-- Logs begin at Wed 2022-07-27 10:05:11 UTC, end at Wed 2022-07-27 11:15:41 UTC. -- Jul 27 10:33:26.672797 dv-cinder-850840-3-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jul 27 10:33:26.683584 dv-cinder-850840-3-iscsi devstack@keystone.service[86814]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jul 27 10:33:26.685782 dv-cinder-850840-3-iscsi devstack@keystone.service[86814]: *** Starting uWSGI 2.0.18-debian (64bit) on [Wed Jul 27 10:33:26 2022] *** Jul 27 10:33:26.685782 dv-cinder-850840-3-iscsi devstack@keystone.service[86814]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jul 27 10:33:26.685782 dv-cinder-850840-3-iscsi devstack@keystone.service[86814]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jul 27 10:33:26.685782 dv-cinder-850840-3-iscsi devstack@keystone.service[86814]: nodename: dv-cinder-850840-3-iscsi Jul 27 10:33:26.685782 dv-cinder-850840-3-iscsi devstack@keystone.service[86814]: machine: x86_64 Jul 27 10:33:26.685782 dv-cinder-850840-3-iscsi devstack@keystone.service[86814]: clock source: unix Jul 27 10:33:26.685782 dv-cinder-850840-3-iscsi devstack@keystone.service[86814]: pcre jit disabled Jul 27 10:33:26.685782 dv-cinder-850840-3-iscsi devstack@keystone.service[86814]: detected number of CPU cores: 4 Jul 27 10:33:26.685782 dv-cinder-850840-3-iscsi devstack@keystone.service[86814]: current working directory: / Jul 27 10:33:26.685782 dv-cinder-850840-3-iscsi devstack@keystone.service[86814]: detected binary path: /usr/bin/uwsgi-core Jul 27 10:33:26.686388 dv-cinder-850840-3-iscsi devstack@keystone.service[86814]: your processes number limit is 47680 Jul 27 10:33:26.686388 dv-cinder-850840-3-iscsi devstack@keystone.service[86814]: your memory page size is 4096 bytes Jul 27 10:33:26.686388 dv-cinder-850840-3-iscsi devstack@keystone.service[86814]: detected max file descriptor number: 2048 Jul 27 10:33:26.686388 dv-cinder-850840-3-iscsi devstack@keystone.service[86814]: lock engine: pthread robust mutexes Jul 27 10:33:26.686388 dv-cinder-850840-3-iscsi devstack@keystone.service[86814]: thunder lock: enabled Jul 27 10:33:26.686388 dv-cinder-850840-3-iscsi devstack@keystone.service[86814]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jul 27 10:33:26.686388 dv-cinder-850840-3-iscsi devstack@keystone.service[86814]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Jul 27 10:33:26.726122 dv-cinder-850840-3-iscsi devstack@keystone.service[86814]: Python main interpreter initialized at 0x55a1644319c0 Jul 27 10:33:26.726122 dv-cinder-850840-3-iscsi devstack@keystone.service[86814]: python threads support enabled Jul 27 10:33:26.726122 dv-cinder-850840-3-iscsi devstack@keystone.service[86814]: your server socket listen backlog is limited to 100 connections Jul 27 10:33:26.726122 dv-cinder-850840-3-iscsi devstack@keystone.service[86814]: your mercy for graceful operations on workers is 90 seconds Jul 27 10:33:26.726510 dv-cinder-850840-3-iscsi devstack@keystone.service[86814]: mapped 403077 bytes (393 KB) for 2 cores Jul 27 10:33:26.726585 dv-cinder-850840-3-iscsi devstack@keystone.service[86814]: *** Operational MODE: preforking *** Jul 27 10:33:26.726917 dv-cinder-850840-3-iscsi devstack@keystone.service[86814]: *** uWSGI is running in multiple interpreter mode *** Jul 27 10:33:26.726917 dv-cinder-850840-3-iscsi devstack@keystone.service[86814]: spawned uWSGI master process (pid: 86814) Jul 27 10:33:26.727130 dv-cinder-850840-3-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jul 27 10:33:26.727715 dv-cinder-850840-3-iscsi devstack@keystone.service[86814]: spawned uWSGI worker 1 (pid: 86827, cores: 1) Jul 27 10:33:26.728545 dv-cinder-850840-3-iscsi devstack@keystone.service[86814]: spawned uWSGI worker 2 (pid: 86828, cores: 1) Jul 27 10:33:26.728706 dv-cinder-850840-3-iscsi devstack@keystone.service[86814]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jul 27 10:33:32.265931 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 27 10:33:32.265931 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: from cryptography.utils import int_from_bytes Jul 27 10:33:32.266724 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 27 10:33:32.266724 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: from cryptography.utils import int_from_bytes Jul 27 10:33:32.287082 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 27 10:33:32.287082 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: from cryptography.utils import int_from_bytes Jul 27 10:33:32.287599 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 27 10:33:32.287599 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: from cryptography.utils import int_from_bytes Jul 27 10:33:32.317251 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 27 10:33:32.317447 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 27 10:33:32.317640 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] command line args: [] {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 27 10:33:32.317919 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 27 10:33:32.318096 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] ================================================================================ {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 27 10:33:32.318353 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] admin_token = **** {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 10:33:32.318618 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] config_dir = [] {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 10:33:32.318812 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 10:33:32.318996 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] config_source = [] {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 10:33:32.319176 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] debug = True {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 10:33:32.319381 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 10:33:32.319873 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 10:33:32.319967 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] insecure_debug = False {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 10:33:32.320157 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 10:33:32.320406 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 10:33:32.320637 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] list_limit = None {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 10:33:32.320818 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] log_config_append = None {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 10:33:32.320997 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 10:33:32.321215 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] log_dir = None {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 10:33:32.321350 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] log_file = None {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 10:33:32.321612 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 10:33:32.321864 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 10:33:32.322112 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 10:33:32.322359 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 10:33:32.322535 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 10:33:32.322735 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 10:33:32.322983 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 10:33:32.323231 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 10:33:32.323465 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 10:33:32.323699 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 10:33:32.323931 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 10:33:32.324164 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 10:33:32.324411 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 10:33:32.324659 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] notification_format = cadf {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 10:33:32.324931 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 10:33:32.325430 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.36/identity {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 10:33:32.325614 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] publish_errors = False {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 10:33:32.325850 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 10:33:32.326085 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 10:33:32.326265 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 10:33:32.326513 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 10:33:32.326771 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 10:33:32.327009 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] standard_threads = False {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 10:33:32.327240 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] strict_password_check = False {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 10:33:32.327484 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 10:33:32.327661 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] use_eventlog = False {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 10:33:32.327836 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] use_journal = False {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 10:33:32.328010 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] use_json = False {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 10:33:32.328183 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] use_stderr = False {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 10:33:32.328357 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] use_syslog = False {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 10:33:32.328588 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] watch_log_file = False {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 10:33:32.328844 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.329084 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.329348 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.329587 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.329838 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.330075 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.330326 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.330563 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.330842 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.331092 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.331343 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.331579 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.331825 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.332062 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.332310 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.332561 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.332802 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.333053 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.333308 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.333555 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.333794 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.334047 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.334311 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.334565 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.334848 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.335082 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.335314 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.335545 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.335781 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.336028 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.336275 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.336507 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.336686 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 27 10:33:32.336774 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.336891 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 27 10:33:32.336970 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.337593 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] command line args: [] {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 27 10:33:32.337724 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.337876 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 27 10:33:32.337964 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.338058 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] ================================================================================ {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 27 10:33:32.338200 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.338325 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] admin_token = **** {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 10:33:32.338429 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.338574 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] config_dir = [] {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 10:33:32.338685 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.338789 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 10:33:32.338913 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.338991 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] config_source = [] {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 10:33:32.339137 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] debug = True {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 10:33:32.339225 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.339350 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 10:33:32.339458 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.339592 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 10:33:32.339677 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.339759 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] insecure_debug = False {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 10:33:32.339832 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.340010 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 10:33:32.340090 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.340261 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 10:33:32.340345 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.340494 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] list_limit = None {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 10:33:32.340573 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.340675 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] log_config_append = None {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 10:33:32.340753 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.340854 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 10:33:32.340939 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.341019 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] log_dir = None {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 10:33:32.341171 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.341249 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] log_file = None {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 10:33:32.341409 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.341488 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 10:33:32.341657 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 10:33:32.341742 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.341888 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.341971 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 10:33:32.342126 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.342211 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 10:33:32.342298 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 10:33:32.342376 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.342470 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 10:33:32.342589 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.342742 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 10:33:32.342848 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.343009 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 10:33:32.343090 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.343214 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 10:33:32.343293 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.343452 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 10:33:32.343536 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.343688 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 10:33:32.343840 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.343917 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 10:33:32.343987 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.344161 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 10:33:32.344238 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.344409 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] notification_format = cadf {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 10:33:32.344496 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.344677 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.344755 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 10:33:32.344911 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.345148 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.345233 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.36/identity {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 10:33:32.345324 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] publish_errors = False {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 10:33:32.345402 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.345559 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 10:33:32.345652 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.345794 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 10:33:32.345878 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.345973 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 10:33:32.346102 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.346222 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 10:33:32.346348 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.346459 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 10:33:32.346586 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.346719 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] standard_threads = False {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 10:33:32.346846 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.346954 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] strict_password_check = False {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 10:33:32.347084 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.347199 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 10:33:32.347330 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] cache.enabled = True {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.347413 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] use_eventlog = False {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 10:33:32.347540 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] use_journal = False {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 10:33:32.347616 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.347757 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] use_json = False {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 10:33:32.347831 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.347901 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] use_stderr = False {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 10:33:32.348020 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.348098 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] use_syslog = False {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 10:33:32.348257 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.348348 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] watch_log_file = False {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 10:33:32.348493 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.348571 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.348728 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.348806 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.348969 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.349055 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.349202 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.349281 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.349452 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.349529 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.349690 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.349774 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.349931 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.350009 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.350168 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.350309 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.350422 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.350533 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.350654 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.350766 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.350870 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.350976 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.351103 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.351235 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.351335 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.351414 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.351566 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.351682 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.351797 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.351883 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.352031 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.352110 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.352262 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.352341 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.352503 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.352587 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.352877 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.353468 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.353646 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.353731 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.353884 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.353970 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.354142 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.354229 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.354401 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.354480 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.354670 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.354755 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.354908 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] auth.external = None {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.354992 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.355141 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] auth.mapped = None {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.355220 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.355374 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.355461 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.355610 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.355692 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.355852 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.355930 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] auth.password = None {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.356095 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] auth.token = None {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.356184 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.356336 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.356415 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.356568 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.356646 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] catalog.caching = True {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.356802 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.356890 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.357033 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.357111 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.357283 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.357362 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.357521 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.357607 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.357751 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.357828 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.357988 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] credential.caching = True {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.358066 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.358222 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.358308 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] credential.driver = sql {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.358459 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.358543 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.358717 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.358794 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.358954 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.359044 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.359191 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.359271 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.359424 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.359501 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.359661 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.359748 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.359915 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.359994 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.360149 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.360229 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.360389 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.360475 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.360623 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.360702 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.360862 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.360948 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.361101 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.361254 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.361338 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.361421 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.361574 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.361653 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.361814 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.361894 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] federation.caching = True {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.362055 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.362136 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.362296 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.362376 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] federation.driver = sql {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.362538 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.362680 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.362801 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.362881 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.363035 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.363112 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.363276 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.363354 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.363517 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.363596 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.363755 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.363832 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.363991 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.364081 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.364235 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.364314 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.364482 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.364560 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.364716 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] identity.caching = True {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.364793 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.364955 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.365064 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.365195 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.365276 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.365432 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.365509 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.365670 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.365749 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.365919 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] identity.driver = sql {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.366000 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.366151 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.366240 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.366388 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.366474 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.366645 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.366742 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.366890 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.366970 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.367123 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.367202 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] cache.enabled = True {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.367357 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.367438 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.367592 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.367678 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.367837 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.367917 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.368075 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.368154 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.368314 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.368418 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.368578 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.368658 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.368817 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.368899 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.369055 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.369135 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.369320 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.369400 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.369562 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.369644 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.369794 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.369879 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.370032 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.370112 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.370266 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.370347 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.370502 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.370580 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.370765 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.370845 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.371004 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.371084 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.371240 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.371319 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.371471 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.371548 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.371703 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.371783 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.371940 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.372019 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.372174 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.372253 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.372419 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.372498 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.372655 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.372741 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.372884 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.372962 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.373122 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.373204 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.373354 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] ldap.password = **** {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.373433 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.373591 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.373670 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.373828 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.373909 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.374068 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.374147 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] auth.external = None {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.374305 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.374384 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] auth.mapped = None {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.374527 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.374622 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.374806 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.374888 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.375055 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.375135 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] auth.password = None {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.375288 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.375369 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] auth.token = None {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.375520 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.375599 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.375756 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] catalog.caching = True {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.375834 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.376004 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.376083 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.376239 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.376316 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.376475 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.376561 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.376712 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.376791 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.376945 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] ldap.user = None {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.377023 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.377181 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] credential.caching = True {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.377268 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.377435 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] credential.driver = sql {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.377514 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.377667 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.377744 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.377916 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.378003 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.378160 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.378239 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.378404 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.378482 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.378662 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.378757 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.378897 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.378975 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.379133 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.379213 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.379371 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.379459 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.379609 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.379689 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.379842 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.379921 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.380091 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.380179 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.380339 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.380418 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.380586 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.380665 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.380885 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.380972 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.381125 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] federation.caching = True {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.381205 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.381366 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.381445 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.381617 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] federation.driver = sql {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.381705 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.381869 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.381949 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.382105 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.382184 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.382345 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.382434 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.382583 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.382687 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.382854 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.382932 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.383094 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.383184 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.383331 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.383410 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.383581 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.383658 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] policy.driver = sql {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.383811 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.383897 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.384093 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] identity.caching = True {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.384174 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.384332 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.384410 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.384570 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] receipt.caching = True {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.384659 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.385072 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.385175 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.385296 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.385372 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.385552 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] identity.driver = sql {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.385647 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.385787 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.385868 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.386030 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.386110 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.386277 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] resource.caching = True {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.386364 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.386529 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.386637 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.386790 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.386869 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] resource.driver = sql {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.387029 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.387122 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.387267 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.387347 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.387517 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.387603 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.387751 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] revoke.caching = True {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.387838 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.388001 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.388081 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.388239 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.388325 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.388485 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] role.cache_time = None {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.388573 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.388723 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] role.caching = True {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.388803 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.388972 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] role.driver = sql {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.389051 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.389204 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] role.list_limit = None {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.389285 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.389436 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.389515 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.389677 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.389757 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.389911 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.389992 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.390150 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.390230 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.390394 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.390473 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.390664 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.390782 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.390910 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.390992 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.391145 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.391224 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.391380 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.391461 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.391627 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.391707 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.391865 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.391944 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.392110 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.392190 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.392357 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.392436 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.392593 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.392762 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.392851 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.393001 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.393081 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] ldap.password = **** {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.393254 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.393334 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.393503 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.393596 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.393746 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.393825 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.393987 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.394067 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.394222 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.394310 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.394472 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.394552 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.394750 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.394833 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.394987 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.395079 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.395219 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.395298 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.395455 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.395535 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.395688 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.395777 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.395933 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.396012 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.396172 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.396252 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.396411 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.396500 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.396645 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] ldap.user = None {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.396725 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.396880 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.396958 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.397114 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] token.cache_time = None {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.397195 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.397349 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] token.caching = True {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.397429 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.397588 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.397667 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.397838 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] token.provider = fernet {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.397921 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.398076 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.398157 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.398313 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.398391 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.398548 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.398692 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.398813 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.398895 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.399054 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.399134 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.399294 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.399377 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.399531 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.400419 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] trust.driver = sql {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.400419 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.400567 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.400649 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.400725 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.400797 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.400869 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.400983 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.401064 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.401232 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.401315 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.401464 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.401544 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.401708 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.401789 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.401949 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.402030 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.402190 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.402270 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] database.connection = **** {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.402431 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.402521 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.402693 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.402776 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.402942 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.403086 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.403194 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.403274 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.403434 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.403514 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.403668 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.403748 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] policy.driver = sql {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.403905 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.403985 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.404141 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.404220 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.404379 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.404459 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.404613 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.404690 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] receipt.caching = True {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.404858 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.404960 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.405093 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.405214 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.405342 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.405463 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.405577 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.405700 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.405823 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.405942 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.406054 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.406184 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] resource.caching = True {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.406294 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.406440 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.406522 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.406710 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=86827) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 27 10:33:32.406793 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] resource.driver = sql {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.406962 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.407219 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.407474 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.407713 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] revoke.caching = True {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.407965 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.408212 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.408457 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] role.cache_time = None {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.408696 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] role.caching = True {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.408950 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] role.driver = sql {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.409185 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] role.list_limit = None {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.409439 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.409692 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.409943 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.410194 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.410445 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.410724 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.410978 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.411230 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.411465 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.411715 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.411964 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.412215 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.412463 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.412868 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.413113 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.413367 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.413621 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.413873 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.414124 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.414364 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.414640 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.414900 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.415143 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.416413 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.416413 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.417102 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.417102 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.417102 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.417401 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.417650 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.417892 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.418128 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] token.cache_time = None {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.418374 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] token.caching = True {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.418633 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.418891 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] token.provider = fernet {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.419132 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.419393 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.419646 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.419889 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.420136 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.420383 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.420636 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] trust.driver = sql {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.420879 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.421122 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.421359 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.421609 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.421862 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.422096 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.422341 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.422676 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.422949 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] database.connection = **** {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.423198 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.423451 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.423691 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.423935 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.424174 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.424414 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.424652 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.424892 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.425143 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.425383 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.425629 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.425868 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.426123 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.426363 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.426624 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.426868 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.427110 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.427351 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 10:33:32.427521 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=86828) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 27 10:33:32.448573 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=86827) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 10:33:32.448901 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=86827) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 10:33:32.449221 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=86827) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 10:33:32.452466 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=86828) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 10:33:32.452698 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=86828) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 10:33:32.452921 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=86828) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 10:33:32.461350 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=86828) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 10:33:32.464045 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=86827) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 10:33:32.495199 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=86827) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 10:33:32.495431 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=86827) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 10:33:32.496980 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=86828) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 10:33:32.497217 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=86828) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 10:33:32.548912 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=86828) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 10:33:32.549040 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=86827) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 10:33:32.600036 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=86828) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 10:33:32.600177 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=86827) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 10:33:32.600267 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=86828) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 10:33:32.600386 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=86827) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 10:33:32.665433 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=86828) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 10:33:32.665660 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=86828) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 10:33:32.665855 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=86827) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 10:33:32.665945 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=86828) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 10:33:32.666084 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=86827) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 10:33:32.666171 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=86828) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 10:33:32.666298 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=86828) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 10:33:32.666380 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=86827) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 10:33:32.666512 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=86828) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 10:33:32.666624 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=86827) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 10:33:32.666752 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=86827) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 10:33:32.666968 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=86827) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 10:33:32.689513 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=86828) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 10:33:32.689624 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=86827) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 10:33:32.689742 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=86828) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 10:33:32.689830 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=86827) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 10:33:32.689969 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=86828) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 10:33:32.690053 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=86827) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 10:33:32.690203 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=86828) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 10:33:32.690296 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=86827) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 10:33:32.690426 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=86828) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 10:33:32.690509 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=86827) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 10:33:32.690661 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=86827) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 10:33:32.690746 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=86828) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 10:33:32.690875 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=86828) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 10:33:32.690971 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=86827) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 10:33:32.702509 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=86828) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 10:33:32.702630 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=86827) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 10:33:32.715873 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=86828) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 10:33:32.716009 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=86827) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 10:33:32.716390 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=86828) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 10:33:32.716476 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=86827) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 10:33:32.716571 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=86828) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 10:33:32.716649 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=86827) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 10:33:32.716724 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=86828) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 10:33:32.716808 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=86827) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 10:33:32.716885 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=86828) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 10:33:32.716959 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=86827) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 10:33:32.717028 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=86828) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 10:33:32.717100 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=86827) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 10:33:32.717173 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=86828) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 10:33:32.717256 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=86827) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 10:33:32.733956 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=86827) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 10:33:32.734164 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=86827) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 27 10:33:32.734386 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=86828) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 10:33:32.734499 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=86827) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 10:33:32.734619 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=86828) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 27 10:33:32.734752 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=86827) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 27 10:33:32.734940 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=86828) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 10:33:32.735025 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=86827) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 10:33:32.735142 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=86828) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 27 10:33:32.735226 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=86827) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 10:33:32.735411 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=86828) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 10:33:32.735496 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=86827) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 10:33:32.735670 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=86828) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 10:33:32.735752 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=86827) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 10:33:32.735924 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=86828) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 10:33:32.736176 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=86828) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 10:33:32.745336 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=86827) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 10:33:32.745606 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=86827) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 10:33:32.745878 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=86827) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 10:33:32.745985 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=86828) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 10:33:32.746145 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=86827) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 10:33:32.746251 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=86828) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 10:33:32.746454 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=86828) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 10:33:32.746766 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=86828) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 10:33:32.753065 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=86827) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 10:33:32.753689 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=86828) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 10:33:32.757739 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=86827) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 10:33:32.758017 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=86827) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 10:33:32.758295 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=86827) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 10:33:32.758377 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=86828) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 10:33:32.758570 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=86827) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 10:33:32.758738 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=86828) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 10:33:32.758890 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=86827) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 10:33:32.759480 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=86828) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 10:33:32.759681 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=86827) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 10:33:32.760013 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=86828) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 10:33:32.760202 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=86827) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 10:33:32.760294 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=86828) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 10:33:32.760469 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=86827) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 10:33:32.760552 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=86828) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 10:33:32.760730 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=86827) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 10:33:32.760812 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=86828) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 10:33:32.760996 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=86827) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 10:33:32.761080 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=86828) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 10:33:32.761255 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=86827) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 10:33:32.761336 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=86828) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 10:33:32.761560 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=86828) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 10:33:32.761825 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=86828) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 10:33:32.784667 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=86827) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 10:33:32.785382 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=86828) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 10:33:32.786151 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=86827) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 10:33:32.786427 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=86827) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 10:33:32.786908 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=86828) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 10:33:32.787189 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=86828) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 10:33:32.791128 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=86827) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 10:33:32.791402 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=86827) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 10:33:32.791676 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=86827) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 10:33:32.791850 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=86828) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 10:33:32.792129 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=86828) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 10:33:32.792406 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=86828) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 10:33:32.798234 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=86827) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 10:33:32.798510 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=86827) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 10:33:32.799071 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=86828) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 10:33:32.799310 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=86828) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 10:33:32.802879 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=86827) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 10:33:32.803162 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=86827) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 10:33:32.803457 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=86827) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 10:33:32.803583 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=86828) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 10:33:32.803736 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=86827) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 10:33:32.803822 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=86828) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 10:33:32.804005 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=86827) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 10:33:32.804088 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=86828) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 10:33:32.804278 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=86827) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 10:33:32.804362 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=86828) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 10:33:32.804549 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=86827) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 10:33:32.804633 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=86828) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 10:33:32.804816 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=86827) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 10:33:32.804898 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=86828) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 10:33:32.805085 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=86828) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 10:33:32.805368 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=86828) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 10:33:32.821256 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=86827) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 10:33:32.822131 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=86827) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 10:33:32.822453 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=86828) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 10:33:32.822757 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=86828) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 10:33:32.826060 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=86827) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 10:33:32.826712 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=86828) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 10:33:32.829212 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=86827) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 10:33:32.829503 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=86827) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 10:33:32.829893 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=86828) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 10:33:32.830198 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=86828) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 10:33:32.833900 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=86827) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 10:33:32.834647 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=86828) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 10:33:32.837109 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=86827) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 10:33:32.837973 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=86828) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 10:33:32.840281 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=86827) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 10:33:32.840575 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=86827) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 10:33:32.840856 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=86827) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 10:33:32.841154 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=86827) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 10:33:32.841238 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=86828) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 10:33:32.841429 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=86827) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 10:33:32.841525 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=86828) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 10:33:32.841713 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=86827) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 10:33:32.841797 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=86828) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 10:33:32.842030 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=86828) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 10:33:32.842305 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=86828) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 10:33:32.842615 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=86828) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 10:33:32.857670 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=86827) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 10:33:32.857949 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=86827) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 10:33:32.858212 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=86827) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 10:33:32.858469 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=86827) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 10:33:32.858732 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=86828) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 10:33:32.859014 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=86828) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 10:33:32.859278 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=86828) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 10:33:32.859537 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=86828) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 10:33:32.865380 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=86827) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 10:33:32.866490 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=86828) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 10:33:32.866967 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=86827) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 10:33:32.867237 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=86827) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 10:33:32.868068 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=86828) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 10:33:32.868868 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=86828) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 10:33:32.870614 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=86827) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 10:33:32.870896 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=86827) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 10:33:32.871182 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=86827) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 10:33:32.871470 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=86827) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 10:33:32.871747 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=86828) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 10:33:32.871839 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=86827) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 10:33:32.872024 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=86828) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 10:33:32.872316 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=86828) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 10:33:32.872595 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=86828) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 10:33:32.872879 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=86828) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 10:33:32.883189 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=86827) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 10:33:32.883463 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=86827) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 10:33:32.883738 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=86827) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 10:33:32.884612 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=86827) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 10:33:32.885156 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=86827) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 10:33:32.885425 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=86827) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 10:33:32.885518 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=86828) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 10:33:32.885705 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=86827) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 10:33:32.885789 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=86828) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 10:33:32.885998 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=86828) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 10:33:32.886279 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=86828) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 10:33:32.886557 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=86828) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 10:33:32.886852 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=86828) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 10:33:32.887125 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=86828) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 10:33:32.901321 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=86827) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 10:33:32.902895 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=86828) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 10:33:32.904307 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=86827) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 10:33:32.905903 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=86828) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 10:33:32.907277 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=86827) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 10:33:32.908890 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=86828) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 10:33:32.909699 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=86827) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 10:33:32.911279 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=86827) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 10:33:32.911364 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=86828) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 10:33:32.911562 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=86827) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 10:33:32.911838 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=86827) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 10:33:32.912866 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=86828) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 10:33:32.913159 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=86828) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 10:33:32.913477 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=86828) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 10:33:32.918310 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=86827) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 10:33:32.919916 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=86827) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 10:33:32.920117 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=86828) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 10:33:32.921739 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=86828) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 10:33:32.922858 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=86827) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 10:33:32.923144 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=86827) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 10:33:32.923423 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=86827) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 10:33:32.923695 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=86827) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 10:33:32.924790 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=86828) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 10:33:32.925081 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=86828) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 10:33:32.925369 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=86828) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 10:33:32.925655 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=86828) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 10:33:32.933546 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=86827) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 10:33:32.933826 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=86827) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 10:33:32.934099 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=86827) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 10:33:32.934319 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=86828) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 10:33:32.934619 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=86828) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 10:33:32.934906 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=86828) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 10:33:32.941001 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=86827) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 10:33:32.941278 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=86827) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 10:33:32.941552 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=86827) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 10:33:32.941830 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=86828) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 10:33:32.941915 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=86827) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 10:33:32.942094 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=86827) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 10:33:32.942176 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=86828) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 10:33:32.942368 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=86827) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 10:33:32.942453 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=86828) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 10:33:32.942656 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=86827) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 10:33:32.942743 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=86828) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 10:33:32.942919 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=86827) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 10:33:32.943014 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=86828) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 10:33:32.943192 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=86828) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 10:33:32.943298 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=86827) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 10:33:32.943474 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=86828) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 10:33:32.943609 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=86827) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 10:33:32.943741 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=86828) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 10:33:32.943844 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=86827) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 10:33:32.944102 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=86827) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 10:33:32.944183 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=86828) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 10:33:32.944370 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=86827) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 10:33:32.944446 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=86828) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 10:33:32.944627 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=86827) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 10:33:32.944778 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=86828) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 10:33:32.944909 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=86828) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 10:33:32.945183 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=86828) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 10:33:32.945444 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=86828) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 10:33:32.978611 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55a1644319c0 pid: 86827 (default app) Jul 27 10:33:32.979561 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55a1644319c0 pid: 86828 (default app) Jul 27 10:33:32.981938 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7890a481-f539-4bcc-9a37-a7d1c0f30190 None None] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:33:32.982225 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7890a481-f539-4bcc-9a37-a7d1c0f30190 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:33:32.982501 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7890a481-f539-4bcc-9a37-a7d1c0f30190 None None] PATH_INFO: `/v3` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:33:32.986833 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1/1] 10.222.0.36 () {50 vars in 791 bytes} [Wed Jul 27 10:33:32 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:33:40.561895 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1350d3b-deba-43b0-be45-cb9c260fa4e3 None None] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:33:40.562456 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1350d3b-deba-43b0-be45-cb9c260fa4e3 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:33:40.562456 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1350d3b-deba-43b0-be45-cb9c260fa4e3 None None] PATH_INFO: `/` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:33:40.566903 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1/2] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 10:33:40 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 10:33:40.572027 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cd106f-7c39-4f1a-a4a0-5aaabf95d301 None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:33:40.572445 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cd106f-7c39-4f1a-a4a0-5aaabf95d301 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:33:40.572581 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cd106f-7c39-4f1a-a4a0-5aaabf95d301 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:33:40.643227 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: INFO dbcounter [None req-96cd106f-7c39-4f1a-a4a0-5aaabf95d301 None None] Registered counter for database keystone Jul 27 10:33:40.664202 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG oslo_db.sqlalchemy.engines [None req-96cd106f-7c39-4f1a-a4a0-5aaabf95d301 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=86827) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 27 10:33:40.671192 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writer thread running {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 27 10:33:40.848493 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG passlib.handlers.bcrypt [None req-96cd106f-7c39-4f1a-a4a0-5aaabf95d301 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=86827) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 27 10:33:40.848828 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG passlib.handlers.bcrypt [None req-96cd106f-7c39-4f1a-a4a0-5aaabf95d301 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=86827) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 27 10:33:40.903307 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-96cd106f-7c39-4f1a-a4a0-5aaabf95d301 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:33:40.904916 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-96cd106f-7c39-4f1a-a4a0-5aaabf95d301 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:33:41.030066 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 2/3] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 10:33:40 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 459 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Jul 27 10:33:41.148266 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1350d3b-deba-43b0-be45-cb9c260fa4e3 None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:33:41.219273 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: INFO dbcounter [None req-e1350d3b-deba-43b0-be45-cb9c260fa4e3 None None] Registered counter for database keystone Jul 27 10:33:41.238580 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG oslo_db.sqlalchemy.engines [None req-e1350d3b-deba-43b0-be45-cb9c260fa4e3 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=86828) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 27 10:33:41.244799 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writer thread running {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 27 10:33:41.361442 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1350d3b-deba-43b0-be45-cb9c260fa4e3 None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:33:41.365685 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0786fd-4432-4e0a-b24d-151a55cc3bd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:33:41.366325 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0786fd-4432-4e0a-b24d-151a55cc3bd0 admin admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:33:41.366622 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0786fd-4432-4e0a-b24d-151a55cc3bd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:33:41.366910 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0786fd-4432-4e0a-b24d-151a55cc3bd0 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:33:41.374960 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0786fd-4432-4e0a-b24d-151a55cc3bd0 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:33:41.427506 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: WARNING keystone.common.rbac_enforcer.enforcer [None req-ca0786fd-4432-4e0a-b24d-151a55cc3bd0 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 27 10:33:41.427795 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0786fd-4432-4e0a-b24d-151a55cc3bd0 admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:33:41.437193 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 2/4] 10.222.0.36 () {58 vars in 1324 bytes} [Wed Jul 27 10:33:41 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 27 10:33:41.443111 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cd106f-7c39-4f1a-a4a0-5aaabf95d301 None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:33:41.457751 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cd106f-7c39-4f1a-a4a0-5aaabf95d301 None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:33:41.462726 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a662be8c-4d60-4510-b26a-3c738868c5a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:33:41.463896 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a662be8c-4d60-4510-b26a-3c738868c5a5 admin admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:33:41.464123 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a662be8c-4d60-4510-b26a-3c738868c5a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:33:41.464394 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a662be8c-4d60-4510-b26a-3c738868c5a5 admin admin] PATH_INFO: `/v3/projects/eeee704de4024dd1b602d1a289cba407` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:33:41.465876 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a662be8c-4d60-4510-b26a-3c738868c5a5 admin admin] RBAC: Authorizing `identity:get_project(project_id=eeee704de4024dd1b602d1a289cba407)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:33:41.516883 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: WARNING keystone.common.rbac_enforcer.enforcer [None req-a662be8c-4d60-4510-b26a-3c738868c5a5 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 27 10:33:41.517221 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a662be8c-4d60-4510-b26a-3c738868c5a5 admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:33:41.521809 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 3/5] 10.222.0.36 () {56 vars in 1207 bytes} [Wed Jul 27 10:33:41 2022] GET /identity/v3/projects/eeee704de4024dd1b602d1a289cba407 => generated 344 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:33:43.495057 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0c4370-6c41-4afe-8ab5-c6686b085842 None None] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:33:43.495594 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0c4370-6c41-4afe-8ab5-c6686b085842 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:33:43.495594 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0c4370-6c41-4afe-8ab5-c6686b085842 None None] PATH_INFO: `/` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:33:43.499745 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 3/6] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 10:33:43 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 10:33:43.505058 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f819bf0-970b-4117-be34-7b1aa5880805 None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:33:43.505474 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f819bf0-970b-4117-be34-7b1aa5880805 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:33:43.505621 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f819bf0-970b-4117-be34-7b1aa5880805 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:33:43.526539 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-3f819bf0-970b-4117-be34-7b1aa5880805 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:33:43.528093 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-3f819bf0-970b-4117-be34-7b1aa5880805 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:33:43.535895 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 4/7] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 10:33:43 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jul 27 10:33:43.557619 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0c4370-6c41-4afe-8ab5-c6686b085842 None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:33:43.569626 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0c4370-6c41-4afe-8ab5-c6686b085842 None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:33:43.574493 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce6d2ec-20c3-413f-baba-105fb773d33a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:33:43.575135 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce6d2ec-20c3-413f-baba-105fb773d33a admin admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:33:43.575429 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce6d2ec-20c3-413f-baba-105fb773d33a admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:33:43.575708 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce6d2ec-20c3-413f-baba-105fb773d33a admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:33:43.583410 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce6d2ec-20c3-413f-baba-105fb773d33a admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:33:43.585180 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce6d2ec-20c3-413f-baba-105fb773d33a admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:33:43.593922 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 4/8] 10.222.0.36 () {58 vars in 1324 bytes} [Wed Jul 27 10:33:43 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 27 10:33:43.599094 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f819bf0-970b-4117-be34-7b1aa5880805 None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:33:43.608120 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f819bf0-970b-4117-be34-7b1aa5880805 None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:33:43.612506 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f57c94d-5140-41e5-93db-0ee179f373a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:33:43.613142 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f57c94d-5140-41e5-93db-0ee179f373a3 admin admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:33:43.613428 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f57c94d-5140-41e5-93db-0ee179f373a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:33:43.613711 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f57c94d-5140-41e5-93db-0ee179f373a3 admin admin] PATH_INFO: `/v3/users/60b0ea1b90004781b58a0cbfd103ae8f` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:33:43.615117 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f57c94d-5140-41e5-93db-0ee179f373a3 admin admin] RBAC: Authorizing `identity:get_user(user_id=60b0ea1b90004781b58a0cbfd103ae8f)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:33:43.616945 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f57c94d-5140-41e5-93db-0ee179f373a3 admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:33:43.619057 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 5/9] 10.222.0.36 () {56 vars in 1198 bytes} [Wed Jul 27 10:33:43 2022] GET /identity/v3/users/60b0ea1b90004781b58a0cbfd103ae8f => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:33:45.684198 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa031071-91de-45e7-aea0-eeaa0b044def None None] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:33:45.684707 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa031071-91de-45e7-aea0-eeaa0b044def None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:33:45.684805 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa031071-91de-45e7-aea0-eeaa0b044def None None] PATH_INFO: `/` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:33:45.686112 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 5/10] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 10:33:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 10:33:45.691245 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bbc224-b818-49b7-bec6-a37da94872d7 None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:33:45.691635 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bbc224-b818-49b7-bec6-a37da94872d7 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:33:45.691936 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bbc224-b818-49b7-bec6-a37da94872d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:33:45.711849 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-e0bbc224-b818-49b7-bec6-a37da94872d7 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:33:45.713065 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-e0bbc224-b818-49b7-bec6-a37da94872d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:33:45.782943 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5273f83c-0ac1-41db-9d97-c62831511290 None None] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:33:45.783231 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5273f83c-0ac1-41db-9d97-c62831511290 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:33:45.783505 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5273f83c-0ac1-41db-9d97-c62831511290 None None] PATH_INFO: `/` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:33:45.784830 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 6/11] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 10:33:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 10:33:45.789533 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5736c22a-c78e-41fa-abe4-eb9cb25f03ec None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:33:45.789820 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5736c22a-c78e-41fa-abe4-eb9cb25f03ec None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:33:45.790679 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5736c22a-c78e-41fa-abe4-eb9cb25f03ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:33:45.793020 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 6/12] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 10:33:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 10:33:45.815489 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bbc224-b818-49b7-bec6-a37da94872d7 None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:33:45.832131 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG passlib.handlers.bcrypt [None req-5736c22a-c78e-41fa-abe4-eb9cb25f03ec None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=86828) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 27 10:33:45.832559 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG passlib.handlers.bcrypt [None req-5736c22a-c78e-41fa-abe4-eb9cb25f03ec None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=86828) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 27 10:33:45.839356 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:33:45.878773 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-5736c22a-c78e-41fa-abe4-eb9cb25f03ec None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:33:45.880359 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-5736c22a-c78e-41fa-abe4-eb9cb25f03ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:33:45.918423 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bbc224-b818-49b7-bec6-a37da94872d7 None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:33:45.952933 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff2d203-6e67-45d6-990c-97e92bd23e9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:33:45.953596 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff2d203-6e67-45d6-990c-97e92bd23e9d None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:33:45.953885 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff2d203-6e67-45d6-990c-97e92bd23e9d None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:33:45.954171 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff2d203-6e67-45d6-990c-97e92bd23e9d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:33:45.966340 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff2d203-6e67-45d6-990c-97e92bd23e9d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 10:33:45.975239 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application [None req-bff2d203-6e67-45d6-990c-97e92bd23e9d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 10:33:45.975239 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:33:45.975239 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:33:45.975239 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:33:45.975239 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:33:45.975239 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:33:45.975239 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:33:45.975239 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:33:45.975239 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:33:45.975239 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:33:45.975239 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:33:45.975239 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:33:45.975239 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 10:33:45.975239 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 10:33:45.975239 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 10:33:45.975239 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 10:33:45.976089 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 10:33:45.976089 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 10:33:45.976089 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 10:33:45.976089 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 10:33:45.976089 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:33:45.976089 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:33:45.976089 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:33:45.976089 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:33:45.976089 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:33:45.976089 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:33:45.976089 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:33:45.976089 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application with Lock( Jul 27 10:33:45.976089 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:33:45.976089 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:33:45.976089 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:33:45.976089 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:33:45.977007 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:33:45.977007 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:33:45.977007 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:33:45.977007 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:33:45.977007 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 10:33:45.977007 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 10:33:45.977007 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 10:33:45.977007 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 10:33:45.977007 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 10:33:45.977007 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 10:33:45.977007 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 10:33:45.977007 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Jul 27 10:33:45.977007 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 7/13] 10.222.0.36 () {56 vars in 1096 bytes} [Wed Jul 27 10:33:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 10:33:45.980302 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 7/14] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 10:33:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 27 10:33:45.982269 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff2d203-6e67-45d6-990c-97e92bd23e9d None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:33:46.001442 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5736c22a-c78e-41fa-abe4-eb9cb25f03ec None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:33:46.064875 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:33:46.085329 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff2d203-6e67-45d6-990c-97e92bd23e9d None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:33:46.101231 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5736c22a-c78e-41fa-abe4-eb9cb25f03ec None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:33:46.103544 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:33:46.119228 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8699bc9a-6f35-4f99-80e4-3cb54c279df9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:33:46.119850 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8699bc9a-6f35-4f99-80e4-3cb54c279df9 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:33:46.120138 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8699bc9a-6f35-4f99-80e4-3cb54c279df9 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:33:46.120416 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8699bc9a-6f35-4f99-80e4-3cb54c279df9 None admin] PATH_INFO: `/v3/roles` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:33:46.121157 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8699bc9a-6f35-4f99-80e4-3cb54c279df9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86827) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:33:46.121459 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8699bc9a-6f35-4f99-80e4-3cb54c279df9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:33:46.122907 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8699bc9a-6f35-4f99-80e4-3cb54c279df9 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:33:46.135318 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 8/15] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jul 27 10:33:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:33:46.139348 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be22eb7-5a44-4171-98c4-62c3aaab0d6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:33:46.140230 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be22eb7-5a44-4171-98c4-62c3aaab0d6e None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:33:46.140523 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be22eb7-5a44-4171-98c4-62c3aaab0d6e None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:33:46.140803 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be22eb7-5a44-4171-98c4-62c3aaab0d6e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:33:46.141096 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8699bc9a-6f35-4f99-80e4-3cb54c279df9 None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:33:46.208301 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be22eb7-5a44-4171-98c4-62c3aaab0d6e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:33:46.209751 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be22eb7-5a44-4171-98c4-62c3aaab0d6e None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:33:46.240692 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8699bc9a-6f35-4f99-80e4-3cb54c279df9 None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:33:46.311933 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 8/16] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jul 27 10:33:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 27 10:33:46.317844 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be22eb7-5a44-4171-98c4-62c3aaab0d6e None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:33:46.321049 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:33:46.387034 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a13f79-72b8-4201-8ee2-ae30d547fb9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:33:46.387669 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a13f79-72b8-4201-8ee2-ae30d547fb9a None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:33:46.388142 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a13f79-72b8-4201-8ee2-ae30d547fb9a None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:33:46.388419 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a13f79-72b8-4201-8ee2-ae30d547fb9a None admin] PATH_INFO: `/v3/users/60b0ea1b90004781b58a0cbfd103ae8f` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:33:46.389586 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a13f79-72b8-4201-8ee2-ae30d547fb9a None admin] RBAC: Authorizing `identity:get_user(user_id=60b0ea1b90004781b58a0cbfd103ae8f)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:33:46.391059 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a13f79-72b8-4201-8ee2-ae30d547fb9a None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:33:46.392643 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 9/17] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Jul 27 10:33:46 2022] GET /identity/v3/users/60b0ea1b90004781b58a0cbfd103ae8f => generated 247 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 10:33:46.398090 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a13f79-72b8-4201-8ee2-ae30d547fb9a None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:33:46.412976 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:33:46.417335 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be22eb7-5a44-4171-98c4-62c3aaab0d6e None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:33:46.453126 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5b96f6-c58a-472a-86c9-ee172b291190 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:33:46.453979 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5b96f6-c58a-472a-86c9-ee172b291190 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:33:46.454268 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5b96f6-c58a-472a-86c9-ee172b291190 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:33:46.454547 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5b96f6-c58a-472a-86c9-ee172b291190 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:33:46.472662 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5b96f6-c58a-472a-86c9-ee172b291190 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:33:46.474068 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5b96f6-c58a-472a-86c9-ee172b291190 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:33:46.492063 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application [None req-0f5b96f6-c58a-472a-86c9-ee172b291190 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 10:33:46.492063 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:33:46.492063 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:33:46.492063 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:33:46.492063 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:33:46.492063 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:33:46.492063 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:33:46.492063 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:33:46.492063 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:33:46.492063 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:33:46.492063 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:33:46.492063 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:33:46.492063 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 10:33:46.492063 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 10:33:46.492063 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 10:33:46.493546 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 10:33:46.493546 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:33:46.493546 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:33:46.493546 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:33:46.493546 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:33:46.493546 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:33:46.493546 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:33:46.493546 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:33:46.493546 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application with Lock( Jul 27 10:33:46.493546 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:33:46.493546 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:33:46.493546 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:33:46.493546 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:33:46.493546 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:33:46.493546 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:33:46.493546 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:33:46.493546 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:33:46.494359 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 10:33:46.494359 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 10:33:46.494359 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 10:33:46.494359 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Jul 27 10:33:46.494359 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 9/18] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 10:33:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 10:33:46.498423 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a13f79-72b8-4201-8ee2-ae30d547fb9a None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:33:46.499418 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5b96f6-c58a-472a-86c9-ee172b291190 None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:33:46.532801 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e386f06-53aa-4859-ae57-bed7d0fe95f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:33:46.533439 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e386f06-53aa-4859-ae57-bed7d0fe95f6 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:33:46.533721 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e386f06-53aa-4859-ae57-bed7d0fe95f6 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:33:46.534023 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e386f06-53aa-4859-ae57-bed7d0fe95f6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:33:46.535203 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e386f06-53aa-4859-ae57-bed7d0fe95f6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:33:46.536633 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e386f06-53aa-4859-ae57-bed7d0fe95f6 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:33:46.538275 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 10/19] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 10:33:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 10:33:46.544011 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e386f06-53aa-4859-ae57-bed7d0fe95f6 None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:33:46.556284 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:33:46.600843 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5b96f6-c58a-472a-86c9-ee172b291190 None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:33:46.624936 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:33:46.635862 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739fbea8-ecb3-43ae-a76a-d272d237dc1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:33:46.636489 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739fbea8-ecb3-43ae-a76a-d272d237dc1d None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:33:46.636776 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739fbea8-ecb3-43ae-a76a-d272d237dc1d None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:33:46.637049 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739fbea8-ecb3-43ae-a76a-d272d237dc1d None admin] PATH_INFO: `/v3/domains` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:33:46.637750 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739fbea8-ecb3-43ae-a76a-d272d237dc1d None admin] RBAC: Adding query filter params (name=Default) {{(pid=86828) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:33:46.638058 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739fbea8-ecb3-43ae-a76a-d272d237dc1d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:33:46.639482 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739fbea8-ecb3-43ae-a76a-d272d237dc1d None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:33:46.644649 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e386f06-53aa-4859-ae57-bed7d0fe95f6 None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:33:46.656690 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 10/20] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 10:33:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 10:33:46.679398 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689f3cf3-4f05-4bf8-955c-99da278df26c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:33:46.680040 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689f3cf3-4f05-4bf8-955c-99da278df26c None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:33:46.680321 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689f3cf3-4f05-4bf8-955c-99da278df26c None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:33:46.681198 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689f3cf3-4f05-4bf8-955c-99da278df26c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:33:46.681993 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689f3cf3-4f05-4bf8-955c-99da278df26c None admin] RBAC: Adding query filter params (role.id=21156560eb6b436bb567a40b4976f3f4, scope.domain.id=default, user.id=60b0ea1b90004781b58a0cbfd103ae8f) {{(pid=86827) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:33:46.682303 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689f3cf3-4f05-4bf8-955c-99da278df26c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:33:46.683837 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689f3cf3-4f05-4bf8-955c-99da278df26c None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:33:46.691511 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 11/21] 10.222.0.36 () {56 vars in 1322 bytes} [Wed Jul 27 10:33:46 2022] GET /identity/v3/role_assignments?user.id=60b0ea1b90004781b58a0cbfd103ae8f&scope.domain.id=default&role.id=21156560eb6b436bb567a40b4976f3f4 => generated 233 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 10:33:48.961448 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff363757-20c9-480b-aedb-3054c4658606 None None] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:33:48.962194 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff363757-20c9-480b-aedb-3054c4658606 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:33:48.962782 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff363757-20c9-480b-aedb-3054c4658606 None None] PATH_INFO: `/` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:33:48.964352 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 11/22] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 10:33:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 10:33:48.969767 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df0570e-4ba7-41a2-9fef-057ec295be75 None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:33:48.970584 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df0570e-4ba7-41a2-9fef-057ec295be75 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:33:48.971081 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df0570e-4ba7-41a2-9fef-057ec295be75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:33:49.002072 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-5df0570e-4ba7-41a2-9fef-057ec295be75 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:33:49.003629 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-5df0570e-4ba7-41a2-9fef-057ec295be75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:33:49.086454 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 12/23] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 10:33:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Jul 27 10:33:49.121678 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff363757-20c9-480b-aedb-3054c4658606 None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:33:49.132573 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7460330-43a3-4602-93e2-e009a17e8bc3 None None] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:33:49.132877 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7460330-43a3-4602-93e2-e009a17e8bc3 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:33:49.133149 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7460330-43a3-4602-93e2-e009a17e8bc3 None None] PATH_INFO: `/` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:33:49.134552 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 13/24] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 10:33:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 10:33:49.139726 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5e7ce4-67ff-4739-9f46-f395d91fa75e None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:33:49.140018 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5e7ce4-67ff-4739-9f46-f395d91fa75e None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:33:49.140299 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5e7ce4-67ff-4739-9f46-f395d91fa75e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:33:49.174626 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-ab5e7ce4-67ff-4739-9f46-f395d91fa75e None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:33:49.174626 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-ab5e7ce4-67ff-4739-9f46-f395d91fa75e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:33:49.299685 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff363757-20c9-480b-aedb-3054c4658606 None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:33:49.323079 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:33:49.354503 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c70e3d-6220-4e5a-ae76-453025fb9e2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:33:49.355544 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c70e3d-6220-4e5a-ae76-453025fb9e2a None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:33:49.356026 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c70e3d-6220-4e5a-ae76-453025fb9e2a None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:33:49.356494 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c70e3d-6220-4e5a-ae76-453025fb9e2a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:33:49.360114 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 14/25] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 10:33:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 27 10:33:49.361601 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e22e82a-d1aa-49b1-9a43-5d217ae97538 None None] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:33:49.361902 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e22e82a-d1aa-49b1-9a43-5d217ae97538 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:33:49.362187 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e22e82a-d1aa-49b1-9a43-5d217ae97538 None None] PATH_INFO: `/` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:33:49.363619 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 15/26] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 10:33:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 10:33:49.368975 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c70e3d-6220-4e5a-ae76-453025fb9e2a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:33:49.370017 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa86cce8-2736-4314-8816-f15a028613f3 None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:33:49.370308 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa86cce8-2736-4314-8816-f15a028613f3 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:33:49.370592 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa86cce8-2736-4314-8816-f15a028613f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:33:49.371379 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c70e3d-6220-4e5a-ae76-453025fb9e2a None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:33:49.388247 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application [None req-98c70e3d-6220-4e5a-ae76-453025fb9e2a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 10:33:49.388247 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:33:49.388247 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:33:49.388247 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:33:49.388247 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:33:49.388247 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:33:49.388247 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:33:49.388247 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:33:49.388247 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:33:49.388247 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:33:49.388247 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:33:49.388247 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:33:49.388247 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 10:33:49.388247 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 10:33:49.388247 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 10:33:49.389112 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 10:33:49.389112 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:33:49.389112 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:33:49.389112 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:33:49.389112 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:33:49.389112 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:33:49.389112 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:33:49.389112 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:33:49.389112 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application with Lock( Jul 27 10:33:49.389112 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:33:49.389112 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:33:49.389112 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:33:49.389112 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:33:49.389112 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:33:49.389112 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:33:49.389112 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:33:49.389112 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:33:49.389895 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 10:33:49.389895 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 10:33:49.389895 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 10:33:49.389895 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Jul 27 10:33:49.390103 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:33:49.392634 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 12/27] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 10:33:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 10:33:49.397519 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c70e3d-6220-4e5a-ae76-453025fb9e2a None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:33:49.419701 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-fa86cce8-2736-4314-8816-f15a028613f3 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:33:49.420918 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-fa86cce8-2736-4314-8816-f15a028613f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:33:49.510823 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c70e3d-6220-4e5a-ae76-453025fb9e2a None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:33:49.567979 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e0a7ce-7854-42db-939f-1a13affca8f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:33:49.568994 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e0a7ce-7854-42db-939f-1a13affca8f9 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:33:49.569476 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e0a7ce-7854-42db-939f-1a13affca8f9 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:33:49.569971 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e0a7ce-7854-42db-939f-1a13affca8f9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:33:49.611115 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e0a7ce-7854-42db-939f-1a13affca8f9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 10:33:49.628730 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 16/28] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 10:33:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 10:33:49.629908 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa86cce8-2736-4314-8816-f15a028613f3 None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:33:49.801593 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa86cce8-2736-4314-8816-f15a028613f3 None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:33:49.816241 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application [None req-47e0a7ce-7854-42db-939f-1a13affca8f9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 10:33:49.816241 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:33:49.816241 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:33:49.816241 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:33:49.816241 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:33:49.816241 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:33:49.816241 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:33:49.816241 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:33:49.816241 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:33:49.816241 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:33:49.816241 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:33:49.816241 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:33:49.816241 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 10:33:49.816241 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 10:33:49.816241 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 10:33:49.816241 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 10:33:49.817157 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 10:33:49.817157 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 10:33:49.817157 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 10:33:49.817157 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 10:33:49.817157 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:33:49.817157 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:33:49.817157 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:33:49.817157 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:33:49.817157 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:33:49.817157 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:33:49.817157 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:33:49.817157 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application with Lock( Jul 27 10:33:49.817157 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:33:49.817157 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:33:49.817157 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:33:49.817157 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:33:49.817913 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:33:49.817913 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:33:49.817913 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:33:49.817913 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:33:49.817913 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 10:33:49.817913 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 10:33:49.817913 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 10:33:49.817913 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 10:33:49.817913 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 10:33:49.817913 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 10:33:49.817913 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 10:33:49.817913 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Jul 27 10:33:49.817913 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 13/29] 10.222.0.36 () {56 vars in 1096 bytes} [Wed Jul 27 10:33:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 421 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 10:33:49.818564 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e0a7ce-7854-42db-939f-1a13affca8f9 None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:33:49.851062 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbfd0db-729b-448e-9272-83afd886e05c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:33:49.851697 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbfd0db-729b-448e-9272-83afd886e05c None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:33:49.851978 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbfd0db-729b-448e-9272-83afd886e05c None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:33:49.852251 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbfd0db-729b-448e-9272-83afd886e05c None admin] PATH_INFO: `/v3/domains` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:33:49.852980 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbfd0db-729b-448e-9272-83afd886e05c None admin] RBAC: Adding query filter params (name=Default) {{(pid=86827) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:33:49.853285 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbfd0db-729b-448e-9272-83afd886e05c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:33:49.854696 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbfd0db-729b-448e-9272-83afd886e05c None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:33:49.858890 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:33:49.872641 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:33:49.886046 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 17/30] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 10:33:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:33:49.886971 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbfd0db-729b-448e-9272-83afd886e05c None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:33:49.964651 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e0a7ce-7854-42db-939f-1a13affca8f9 None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:33:50.002149 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d55757c-a76f-4dc8-a709-31562a82b85a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:33:50.003137 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d55757c-a76f-4dc8-a709-31562a82b85a None admin] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:33:50.003601 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d55757c-a76f-4dc8-a709-31562a82b85a None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:33:50.004082 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d55757c-a76f-4dc8-a709-31562a82b85a None admin] PATH_INFO: `/v3/roles` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:33:50.004977 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d55757c-a76f-4dc8-a709-31562a82b85a None admin] RBAC: Authorizing `identity:create_role()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:33:50.006808 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d55757c-a76f-4dc8-a709-31562a82b85a None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:33:50.013092 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbfd0db-729b-448e-9272-83afd886e05c None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:33:50.049891 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead7c3be-1d43-4a3d-af02-66d20b704893 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:33:50.050837 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead7c3be-1d43-4a3d-af02-66d20b704893 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:33:50.051265 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead7c3be-1d43-4a3d-af02-66d20b704893 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:33:50.051697 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead7c3be-1d43-4a3d-af02-66d20b704893 None admin] PATH_INFO: `/v3/roles` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:33:50.052557 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead7c3be-1d43-4a3d-af02-66d20b704893 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86827) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:33:50.053003 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead7c3be-1d43-4a3d-af02-66d20b704893 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:33:50.054511 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead7c3be-1d43-4a3d-af02-66d20b704893 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:33:50.063728 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 18/31] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jul 27 10:33:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:33:50.065200 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e7dc7a-9e83-4711-8037-bcc9d1a2f9a5 None None] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:33:50.065709 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e7dc7a-9e83-4711-8037-bcc9d1a2f9a5 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:33:50.066196 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e7dc7a-9e83-4711-8037-bcc9d1a2f9a5 None None] PATH_INFO: `/` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:33:50.067677 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 19/32] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 10:33:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 10:33:50.068818 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e7dc7a-9e83-4711-8037-bcc9d1a2f9a5 None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:33:50.130508 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:33:50.161910 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 14/33] 10.222.0.36 () {60 vars in 1131 bytes} [Wed Jul 27 10:33:49 2022] POST /identity/v3/roles => generated 219 bytes in 343 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 10:33:50.163227 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d55757c-a76f-4dc8-a709-31562a82b85a None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:33:50.179734 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e7dc7a-9e83-4711-8037-bcc9d1a2f9a5 None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:33:50.219048 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7250468-4296-4c7a-a290-98bfa564dad6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:33:50.219961 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7250468-4296-4c7a-a290-98bfa564dad6 None admin] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:33:50.220390 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7250468-4296-4c7a-a290-98bfa564dad6 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:33:50.220808 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7250468-4296-4c7a-a290-98bfa564dad6 None admin] PATH_INFO: `/v3/projects` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:33:50.221691 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7250468-4296-4c7a-a290-98bfa564dad6 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:33:50.223419 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7250468-4296-4c7a-a290-98bfa564dad6 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:33:50.275621 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:33:50.292070 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d55757c-a76f-4dc8-a709-31562a82b85a None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:33:50.333030 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea5c727-d37f-4fbf-b16c-ba5f7ae9ba99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:33:50.333953 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea5c727-d37f-4fbf-b16c-ba5f7ae9ba99 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:33:50.334398 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea5c727-d37f-4fbf-b16c-ba5f7ae9ba99 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:33:50.334856 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea5c727-d37f-4fbf-b16c-ba5f7ae9ba99 None admin] PATH_INFO: `/v3/users/60b0ea1b90004781b58a0cbfd103ae8f` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:33:50.337033 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea5c727-d37f-4fbf-b16c-ba5f7ae9ba99 None admin] RBAC: Authorizing `identity:get_user(user_id=60b0ea1b90004781b58a0cbfd103ae8f)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:33:50.338836 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea5c727-d37f-4fbf-b16c-ba5f7ae9ba99 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:33:50.340795 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 15/34] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Jul 27 10:33:50 2022] GET /identity/v3/users/60b0ea1b90004781b58a0cbfd103ae8f => generated 247 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:33:50.342401 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f69a02-2c3d-4bec-9595-d45da13237f1 None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:33:50.342867 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f69a02-2c3d-4bec-9595-d45da13237f1 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:33:50.343290 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f69a02-2c3d-4bec-9595-d45da13237f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:33:50.363559 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-95f69a02-2c3d-4bec-9595-d45da13237f1 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:33:50.365404 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-95f69a02-2c3d-4bec-9595-d45da13237f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:33:50.376418 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 20/35] 10.222.0.36 () {60 vars in 1141 bytes} [Wed Jul 27 10:33:50 2022] POST /identity/v3/projects => generated 301 bytes in 308 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 10:33:50.378267 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299eb6a7-d047-46fa-a765-22d692dce6e1 None None] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:33:50.378641 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299eb6a7-d047-46fa-a765-22d692dce6e1 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:33:50.379066 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299eb6a7-d047-46fa-a765-22d692dce6e1 None None] PATH_INFO: `/` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:33:50.380671 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 21/36] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 10:33:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 10:33:50.381562 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299eb6a7-d047-46fa-a765-22d692dce6e1 None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:33:50.452222 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 16/37] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 10:33:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 10:33:50.454152 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c0ede1-b229-43e0-aaf6-71f4e86b5c66 None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:33:50.454629 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c0ede1-b229-43e0-aaf6-71f4e86b5c66 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:33:50.458636 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c0ede1-b229-43e0-aaf6-71f4e86b5c66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:33:50.486524 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-36c0ede1-b229-43e0-aaf6-71f4e86b5c66 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:33:50.492315 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-36c0ede1-b229-43e0-aaf6-71f4e86b5c66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:33:50.508901 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299eb6a7-d047-46fa-a765-22d692dce6e1 None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:33:50.520699 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:33:50.555042 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09d75e2-1c48-43d6-9b58-0ff419c5620a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:33:50.556503 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09d75e2-1c48-43d6-9b58-0ff419c5620a None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:33:50.556939 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09d75e2-1c48-43d6-9b58-0ff419c5620a None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:33:50.557285 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09d75e2-1c48-43d6-9b58-0ff419c5620a None admin] PATH_INFO: `/v3/domains/default` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:33:50.559851 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09d75e2-1c48-43d6-9b58-0ff419c5620a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:33:50.561883 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09d75e2-1c48-43d6-9b58-0ff419c5620a None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:33:50.563790 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 22/38] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 10:33:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:33:50.565765 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09d75e2-1c48-43d6-9b58-0ff419c5620a None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:33:50.588480 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 17/39] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 10:33:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 10:33:50.589573 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c0ede1-b229-43e0-aaf6-71f4e86b5c66 None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:33:50.610495 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:33:50.690396 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09d75e2-1c48-43d6-9b58-0ff419c5620a None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:33:50.707645 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c0ede1-b229-43e0-aaf6-71f4e86b5c66 None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:33:50.724586 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0511cde-81b5-4473-b7d8-074fc25629d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:33:50.725211 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0511cde-81b5-4473-b7d8-074fc25629d8 None admin] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:33:50.725499 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0511cde-81b5-4473-b7d8-074fc25629d8 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:33:50.725779 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0511cde-81b5-4473-b7d8-074fc25629d8 None admin] PATH_INFO: `/v3/roles` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:33:50.726496 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0511cde-81b5-4473-b7d8-074fc25629d8 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:33:50.728421 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0511cde-81b5-4473-b7d8-074fc25629d8 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:33:50.742035 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7469e483-15e4-47d2-ad95-1b0042e553f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:33:50.742774 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7469e483-15e4-47d2-ad95-1b0042e553f7 None admin] REQUEST_METHOD: `PUT` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:33:50.743610 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7469e483-15e4-47d2-ad95-1b0042e553f7 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:33:50.743899 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7469e483-15e4-47d2-ad95-1b0042e553f7 None admin] PATH_INFO: `/v3/domains/default/users/60b0ea1b90004781b58a0cbfd103ae8f/roles/21156560eb6b436bb567a40b4976f3f4` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:33:50.744330 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 23/40] 10.222.0.36 () {60 vars in 1131 bytes} [Wed Jul 27 10:33:50 2022] POST /identity/v3/roles => generated 225 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 10:33:50.745113 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0511cde-81b5-4473-b7d8-074fc25629d8 None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:33:50.751193 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7469e483-15e4-47d2-ad95-1b0042e553f7 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=60b0ea1b90004781b58a0cbfd103ae8f, role_id=21156560eb6b436bb567a40b4976f3f4)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:33:50.752710 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7469e483-15e4-47d2-ad95-1b0042e553f7 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:33:50.775566 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 18/41] 10.222.0.36 () {58 vars in 1361 bytes} [Wed Jul 27 10:33:50 2022] PUT /identity/v3/domains/default/users/60b0ea1b90004781b58a0cbfd103ae8f/roles/21156560eb6b436bb567a40b4976f3f4 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:33:50.776664 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd89bb41-d3dd-42cd-b0d5-8640b44637e0 None None] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:33:50.776963 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd89bb41-d3dd-42cd-b0d5-8640b44637e0 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:33:50.777231 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd89bb41-d3dd-42cd-b0d5-8640b44637e0 None None] PATH_INFO: `/` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:33:50.778831 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 19/42] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 10:33:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 10:33:50.784709 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4280437-66e7-4271-b6a1-fa50ad84ce89 None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:33:50.785234 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4280437-66e7-4271-b6a1-fa50ad84ce89 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:33:50.785708 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4280437-66e7-4271-b6a1-fa50ad84ce89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:33:50.809726 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:33:50.811999 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-c4280437-66e7-4271-b6a1-fa50ad84ce89 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:33:50.813579 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-c4280437-66e7-4271-b6a1-fa50ad84ce89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:33:50.895821 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0511cde-81b5-4473-b7d8-074fc25629d8 None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:33:50.923234 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 20/43] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 10:33:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 10:33:50.943459 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725478cf-ff0e-446a-a242-71d2257dbc73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:33:50.944451 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725478cf-ff0e-446a-a242-71d2257dbc73 None admin] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:33:50.944957 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725478cf-ff0e-446a-a242-71d2257dbc73 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:33:50.945426 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725478cf-ff0e-446a-a242-71d2257dbc73 None admin] PATH_INFO: `/v3/roles` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:33:50.947636 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4280437-66e7-4271-b6a1-fa50ad84ce89 None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:33:50.956349 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:33:50.959518 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725478cf-ff0e-446a-a242-71d2257dbc73 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:33:50.960964 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725478cf-ff0e-446a-a242-71d2257dbc73 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:33:50.981404 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 24/44] 10.222.0.36 () {60 vars in 1131 bytes} [Wed Jul 27 10:33:50 2022] POST /identity/v3/roles => generated 223 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 10:33:51.071465 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4280437-66e7-4271-b6a1-fa50ad84ce89 None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:33:51.108716 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f11ddd7-e653-4d30-bc4e-c4ffe34d6841 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:33:51.109394 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f11ddd7-e653-4d30-bc4e-c4ffe34d6841 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:33:51.109682 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f11ddd7-e653-4d30-bc4e-c4ffe34d6841 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:33:51.109961 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f11ddd7-e653-4d30-bc4e-c4ffe34d6841 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:33:51.111232 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f11ddd7-e653-4d30-bc4e-c4ffe34d6841 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:33:51.112737 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f11ddd7-e653-4d30-bc4e-c4ffe34d6841 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:33:51.114387 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 21/45] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 10:33:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 10:33:51.120314 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725478cf-ff0e-446a-a242-71d2257dbc73 None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:33:51.222816 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:33:51.229147 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725478cf-ff0e-446a-a242-71d2257dbc73 None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:33:51.265881 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf553a57-951f-48ba-9044-7187a77be609 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:33:51.266517 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf553a57-951f-48ba-9044-7187a77be609 None admin] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:33:51.266843 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf553a57-951f-48ba-9044-7187a77be609 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:33:51.267124 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf553a57-951f-48ba-9044-7187a77be609 None admin] PATH_INFO: `/v3/projects` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:33:51.267827 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf553a57-951f-48ba-9044-7187a77be609 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:33:51.269413 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf553a57-951f-48ba-9044-7187a77be609 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:33:51.296171 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 25/46] 10.222.0.36 () {60 vars in 1141 bytes} [Wed Jul 27 10:33:51 2022] POST /identity/v3/projects => generated 312 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 10:33:52.813745 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfd65fd-183c-4858-a17a-d98444c70770 None None] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:33:52.814259 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfd65fd-183c-4858-a17a-d98444c70770 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:33:52.814341 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfd65fd-183c-4858-a17a-d98444c70770 None None] PATH_INFO: `/` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:33:52.815680 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 22/47] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 10:33:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 10:33:52.820750 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc5e5a0-2696-43c8-b8b9-facdcb2a572b None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:33:52.821649 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc5e5a0-2696-43c8-b8b9-facdcb2a572b None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:33:52.821868 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc5e5a0-2696-43c8-b8b9-facdcb2a572b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:33:52.840414 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-ccc5e5a0-2696-43c8-b8b9-facdcb2a572b None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:33:52.841933 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-ccc5e5a0-2696-43c8-b8b9-facdcb2a572b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:33:52.907310 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 26/48] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 10:33:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 10:33:52.929897 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfd65fd-183c-4858-a17a-d98444c70770 None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:33:53.006798 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:33:53.032904 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfd65fd-183c-4858-a17a-d98444c70770 None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:33:53.067009 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37ee0e8-04c1-4a31-a2a1-94ac6a9cf6c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:33:53.067585 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37ee0e8-04c1-4a31-a2a1-94ac6a9cf6c6 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:33:53.067871 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37ee0e8-04c1-4a31-a2a1-94ac6a9cf6c6 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:33:53.068150 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37ee0e8-04c1-4a31-a2a1-94ac6a9cf6c6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:33:53.078106 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37ee0e8-04c1-4a31-a2a1-94ac6a9cf6c6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 10:33:53.083258 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application [None req-f37ee0e8-04c1-4a31-a2a1-94ac6a9cf6c6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 10:33:53.083258 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:33:53.083258 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:33:53.083258 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:33:53.083258 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:33:53.083258 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:33:53.083258 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:33:53.083258 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:33:53.083258 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:33:53.083258 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:33:53.083258 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:33:53.083258 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:33:53.083258 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 10:33:53.083258 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 10:33:53.083258 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 10:33:53.083258 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 10:33:53.084265 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 10:33:53.084265 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 10:33:53.084265 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 10:33:53.084265 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 10:33:53.084265 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:33:53.084265 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:33:53.084265 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:33:53.084265 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:33:53.084265 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:33:53.084265 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:33:53.084265 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:33:53.084265 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application with Lock( Jul 27 10:33:53.084265 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:33:53.084265 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:33:53.084265 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:33:53.084265 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:33:53.085138 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:33:53.085138 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:33:53.085138 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:33:53.085138 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:33:53.085138 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 10:33:53.085138 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 10:33:53.085138 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 10:33:53.085138 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 10:33:53.085138 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 10:33:53.085138 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 10:33:53.085138 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 10:33:53.085138 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Jul 27 10:33:53.085138 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 23/49] 10.222.0.36 () {56 vars in 1096 bytes} [Wed Jul 27 10:33:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 10:33:53.091127 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc5e5a0-2696-43c8-b8b9-facdcb2a572b None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:33:53.169322 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:33:53.192824 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc5e5a0-2696-43c8-b8b9-facdcb2a572b None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:33:53.227194 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227fdf66-8d96-46c7-b212-636be165bb5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:33:53.228442 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227fdf66-8d96-46c7-b212-636be165bb5b None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:33:53.228724 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227fdf66-8d96-46c7-b212-636be165bb5b None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:33:53.229002 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227fdf66-8d96-46c7-b212-636be165bb5b None admin] PATH_INFO: `/v3/roles` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:33:53.229719 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227fdf66-8d96-46c7-b212-636be165bb5b None admin] RBAC: Adding query filter params (name=admin) {{(pid=86827) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:33:53.230025 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227fdf66-8d96-46c7-b212-636be165bb5b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:33:53.231443 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227fdf66-8d96-46c7-b212-636be165bb5b None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:33:53.239652 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 27/50] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jul 27 10:33:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:33:53.245327 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37ee0e8-04c1-4a31-a2a1-94ac6a9cf6c6 None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:33:53.302953 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53211733-d382-4fb1-a964-8c70079a45f4 None None] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:33:53.303250 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53211733-d382-4fb1-a964-8c70079a45f4 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:33:53.303543 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53211733-d382-4fb1-a964-8c70079a45f4 None None] PATH_INFO: `/` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:33:53.304916 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 28/51] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 10:33:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 10:33:53.309906 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dfcfaa-6d3a-4f22-b4ee-90da17b8dc13 None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:33:53.310203 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dfcfaa-6d3a-4f22-b4ee-90da17b8dc13 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:33:53.310490 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dfcfaa-6d3a-4f22-b4ee-90da17b8dc13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:33:53.328431 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-a2dfcfaa-6d3a-4f22-b4ee-90da17b8dc13 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:33:53.329603 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-a2dfcfaa-6d3a-4f22-b4ee-90da17b8dc13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:33:53.362563 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37ee0e8-04c1-4a31-a2a1-94ac6a9cf6c6 None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:33:53.396627 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 29/52] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 10:33:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 27 10:33:53.396775 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90ff8f9-6adc-45c9-9c7f-2aa000256147 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:33:53.397247 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90ff8f9-6adc-45c9-9c7f-2aa000256147 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:33:53.397530 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90ff8f9-6adc-45c9-9c7f-2aa000256147 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:33:53.397811 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90ff8f9-6adc-45c9-9c7f-2aa000256147 None admin] PATH_INFO: `/v3/users/60b0ea1b90004781b58a0cbfd103ae8f` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:33:53.399018 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90ff8f9-6adc-45c9-9c7f-2aa000256147 None admin] RBAC: Authorizing `identity:get_user(user_id=60b0ea1b90004781b58a0cbfd103ae8f)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:33:53.400999 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90ff8f9-6adc-45c9-9c7f-2aa000256147 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:33:53.402612 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 24/53] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Jul 27 10:33:53 2022] GET /identity/v3/users/60b0ea1b90004781b58a0cbfd103ae8f => generated 247 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:33:53.407985 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2dfcfaa-6d3a-4f22-b4ee-90da17b8dc13 None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:33:53.420619 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90ff8f9-6adc-45c9-9c7f-2aa000256147 None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:33:53.423672 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:33:53.476561 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:33:53.507430 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2dfcfaa-6d3a-4f22-b4ee-90da17b8dc13 None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:33:53.520749 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90ff8f9-6adc-45c9-9c7f-2aa000256147 None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:33:53.542319 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33f9983-7ab0-4cc6-b5b1-1629d0d14fd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:33:53.542990 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33f9983-7ab0-4cc6-b5b1-1629d0d14fd6 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:33:53.543275 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33f9983-7ab0-4cc6-b5b1-1629d0d14fd6 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:33:53.543555 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33f9983-7ab0-4cc6-b5b1-1629d0d14fd6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:33:53.544702 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33f9983-7ab0-4cc6-b5b1-1629d0d14fd6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:33:53.546143 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33f9983-7ab0-4cc6-b5b1-1629d0d14fd6 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:33:53.547750 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 30/54] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 10:33:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:33:53.553428 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33f9983-7ab0-4cc6-b5b1-1629d0d14fd6 None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:33:53.554891 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5842fe36-c358-464c-9455-392f03cb9cee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:33:53.555562 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5842fe36-c358-464c-9455-392f03cb9cee None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:33:53.555845 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5842fe36-c358-464c-9455-392f03cb9cee None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:33:53.556127 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5842fe36-c358-464c-9455-392f03cb9cee None admin] PATH_INFO: `/v3/domains/default` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:33:53.557273 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5842fe36-c358-464c-9455-392f03cb9cee None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:33:53.558734 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5842fe36-c358-464c-9455-392f03cb9cee None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:33:53.560302 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 25/55] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 10:33:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 10:33:53.565668 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5842fe36-c358-464c-9455-392f03cb9cee None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:33:53.641334 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:33:53.657465 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33f9983-7ab0-4cc6-b5b1-1629d0d14fd6 None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:33:53.673140 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5842fe36-c358-464c-9455-392f03cb9cee None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:33:53.690413 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501ae5b5-0f13-47a2-8f37-f7a7453b8a7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:33:53.691082 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501ae5b5-0f13-47a2-8f37-f7a7453b8a7d None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:33:53.691372 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501ae5b5-0f13-47a2-8f37-f7a7453b8a7d None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:33:53.691655 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501ae5b5-0f13-47a2-8f37-f7a7453b8a7d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:33:53.692454 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501ae5b5-0f13-47a2-8f37-f7a7453b8a7d None admin] RBAC: Adding query filter params (role.id=21156560eb6b436bb567a40b4976f3f4, scope.domain.id=default, user.id=60b0ea1b90004781b58a0cbfd103ae8f) {{(pid=86827) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:33:53.692770 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501ae5b5-0f13-47a2-8f37-f7a7453b8a7d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:33:53.694222 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501ae5b5-0f13-47a2-8f37-f7a7453b8a7d None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:33:53.700571 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 31/56] 10.222.0.36 () {56 vars in 1322 bytes} [Wed Jul 27 10:33:53 2022] GET /identity/v3/role_assignments?user.id=60b0ea1b90004781b58a0cbfd103ae8f&scope.domain.id=default&role.id=21156560eb6b436bb567a40b4976f3f4 => generated 530 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 10:33:53.706776 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52ac2d6-471d-458b-a7b7-0681eae1501d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:33:53.707431 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52ac2d6-471d-458b-a7b7-0681eae1501d None admin] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:33:53.707719 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52ac2d6-471d-458b-a7b7-0681eae1501d None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:33:53.708001 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52ac2d6-471d-458b-a7b7-0681eae1501d None admin] PATH_INFO: `/v3/projects` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:33:53.708715 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52ac2d6-471d-458b-a7b7-0681eae1501d None admin] RBAC: Authorizing `identity:create_project()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:33:53.710226 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52ac2d6-471d-458b-a7b7-0681eae1501d None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:33:53.742513 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 26/57] 10.222.0.36 () {60 vars in 1140 bytes} [Wed Jul 27 10:33:53 2022] POST /identity/v3/projects => generated 298 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 10:33:55.756568 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728dce97-6daf-48dc-a16d-728077a1e73a None None] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:33:55.757112 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728dce97-6daf-48dc-a16d-728077a1e73a None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:33:55.757112 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728dce97-6daf-48dc-a16d-728077a1e73a None None] PATH_INFO: `/` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:33:55.759139 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 32/58] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 10:33:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 10:33:55.764274 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b816bc-09c6-4e97-99d0-821ab411c89a None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:33:55.764637 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b816bc-09c6-4e97-99d0-821ab411c89a None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:33:55.764838 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b816bc-09c6-4e97-99d0-821ab411c89a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:33:55.782578 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-50b816bc-09c6-4e97-99d0-821ab411c89a None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:33:55.783802 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-50b816bc-09c6-4e97-99d0-821ab411c89a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:33:55.848457 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 27/59] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 10:33:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 10:33:55.872257 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728dce97-6daf-48dc-a16d-728077a1e73a None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:33:55.875165 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:33:55.973383 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728dce97-6daf-48dc-a16d-728077a1e73a None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:33:56.005430 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e091dcd-2fb2-4d08-a584-558a3f3363f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:33:56.006072 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e091dcd-2fb2-4d08-a584-558a3f3363f8 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:33:56.006354 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e091dcd-2fb2-4d08-a584-558a3f3363f8 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:33:56.006708 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e091dcd-2fb2-4d08-a584-558a3f3363f8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:33:56.007797 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e091dcd-2fb2-4d08-a584-558a3f3363f8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:33:56.010072 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e091dcd-2fb2-4d08-a584-558a3f3363f8 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:33:56.011670 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 33/60] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 10:33:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:33:56.017137 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b816bc-09c6-4e97-99d0-821ab411c89a None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:33:56.066742 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:33:56.113151 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b816bc-09c6-4e97-99d0-821ab411c89a None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:33:56.146339 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7825d9f6-f8b3-4571-a0fe-8748b8fe3918 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:33:56.146984 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7825d9f6-f8b3-4571-a0fe-8748b8fe3918 None admin] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:33:56.147273 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7825d9f6-f8b3-4571-a0fe-8748b8fe3918 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:33:56.147559 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7825d9f6-f8b3-4571-a0fe-8748b8fe3918 None admin] PATH_INFO: `/v3/users` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:33:56.148247 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7825d9f6-f8b3-4571-a0fe-8748b8fe3918 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:33:56.149735 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7825d9f6-f8b3-4571-a0fe-8748b8fe3918 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:33:56.176830 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 28/61] 10.222.0.36 () {60 vars in 1132 bytes} [Wed Jul 27 10:33:56 2022] POST /identity/v3/users => generated 275 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 10:33:59.264184 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9058b976-2832-4e97-9ab3-f6381daeefbb None None] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:33:59.265182 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9058b976-2832-4e97-9ab3-f6381daeefbb None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:33:59.265666 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9058b976-2832-4e97-9ab3-f6381daeefbb None None] PATH_INFO: `/` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:33:59.267552 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 34/62] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 10:33:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 10:33:59.272572 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304dadf0-7dbc-4398-a7f7-a0daa648a175 None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:33:59.273381 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304dadf0-7dbc-4398-a7f7-a0daa648a175 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:33:59.273937 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304dadf0-7dbc-4398-a7f7-a0daa648a175 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:33:59.302403 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-304dadf0-7dbc-4398-a7f7-a0daa648a175 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:33:59.303960 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-304dadf0-7dbc-4398-a7f7-a0daa648a175 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:33:59.390228 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 29/63] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 10:33:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 10:33:59.425143 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9058b976-2832-4e97-9ab3-f6381daeefbb None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:33:59.520389 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:33:59.541467 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca6be31-f959-4ed9-8629-f1708150afc5 None None] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:33:59.541768 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca6be31-f959-4ed9-8629-f1708150afc5 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:33:59.542034 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca6be31-f959-4ed9-8629-f1708150afc5 None None] PATH_INFO: `/` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:33:59.543499 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 30/64] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 10:33:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 10:33:59.564083 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b419d90-f2e3-4249-b350-7b6d18afe08f None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:33:59.564388 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b419d90-f2e3-4249-b350-7b6d18afe08f None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:33:59.564646 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b419d90-f2e3-4249-b350-7b6d18afe08f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:33:59.582796 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9058b976-2832-4e97-9ab3-f6381daeefbb None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:33:59.589216 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-5b419d90-f2e3-4249-b350-7b6d18afe08f None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:33:59.595685 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-5b419d90-f2e3-4249-b350-7b6d18afe08f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:33:59.638375 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f3ec2d-c283-420a-bc17-73cb00a38c73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:33:59.639403 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f3ec2d-c283-420a-bc17-73cb00a38c73 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:33:59.639909 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f3ec2d-c283-420a-bc17-73cb00a38c73 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:33:59.640387 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f3ec2d-c283-420a-bc17-73cb00a38c73 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:33:59.646207 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:33:59.653521 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f3ec2d-c283-420a-bc17-73cb00a38c73 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 10:33:59.660577 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application [None req-a5f3ec2d-c283-420a-bc17-73cb00a38c73 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 27 10:33:59.660577 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:33:59.660577 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:33:59.660577 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:33:59.660577 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:33:59.660577 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:33:59.660577 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:33:59.660577 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:33:59.660577 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:33:59.660577 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:33:59.660577 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:33:59.660577 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:33:59.660577 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 10:33:59.660577 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 10:33:59.660577 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 10:33:59.660577 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 10:33:59.661437 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 10:33:59.661437 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 10:33:59.661437 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 10:33:59.661437 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 10:33:59.661437 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:33:59.661437 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:33:59.661437 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:33:59.661437 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:33:59.661437 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:33:59.661437 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:33:59.661437 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:33:59.661437 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application with Lock( Jul 27 10:33:59.661437 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:33:59.661437 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:33:59.661437 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:33:59.661437 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:33:59.662176 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:33:59.662176 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:33:59.662176 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:33:59.662176 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:33:59.662176 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 10:33:59.662176 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 10:33:59.662176 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 10:33:59.662176 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 10:33:59.662176 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 10:33:59.662176 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 10:33:59.662176 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 27 10:33:59.662176 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Jul 27 10:33:59.664677 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 35/65] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jul 27 10:33:59 2022] GET /identity/v3/roles/member => generated 84 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 10:33:59.666757 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70077864-f2f9-4d03-a632-416af2836ce4 None None] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:33:59.667266 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70077864-f2f9-4d03-a632-416af2836ce4 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:33:59.667741 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70077864-f2f9-4d03-a632-416af2836ce4 None None] PATH_INFO: `/` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:33:59.669253 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 36/66] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 10:33:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 10:33:59.671257 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70077864-f2f9-4d03-a632-416af2836ce4 None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:33:59.695892 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 31/67] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 10:33:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 10:33:59.697507 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a66fd7-e18e-401e-ae1d-9b0fdde9548b None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:33:59.697810 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a66fd7-e18e-401e-ae1d-9b0fdde9548b None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:33:59.698094 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a66fd7-e18e-401e-ae1d-9b0fdde9548b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:33:59.720293 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-44a66fd7-e18e-401e-ae1d-9b0fdde9548b None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:33:59.721487 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-44a66fd7-e18e-401e-ae1d-9b0fdde9548b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:33:59.799713 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70077864-f2f9-4d03-a632-416af2836ce4 None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:33:59.800592 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 32/68] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 10:33:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 10:33:59.802102 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1567d31d-0cf7-4072-963f-68c463199138 None None] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:33:59.802466 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1567d31d-0cf7-4072-963f-68c463199138 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:33:59.802891 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1567d31d-0cf7-4072-963f-68c463199138 None None] PATH_INFO: `/` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:33:59.804220 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 33/69] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 10:33:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 10:33:59.804992 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1567d31d-0cf7-4072-963f-68c463199138 None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:33:59.843025 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:33:59.850667 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec4a9e7-3f72-4b86-973d-a285b4d29f57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:33:59.851301 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec4a9e7-3f72-4b86-973d-a285b4d29f57 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:33:59.851584 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec4a9e7-3f72-4b86-973d-a285b4d29f57 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:33:59.851859 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec4a9e7-3f72-4b86-973d-a285b4d29f57 None admin] PATH_INFO: `/v3/roles` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:33:59.852634 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec4a9e7-3f72-4b86-973d-a285b4d29f57 None admin] RBAC: Adding query filter params (name=member) {{(pid=86827) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:33:59.852941 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec4a9e7-3f72-4b86-973d-a285b4d29f57 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:33:59.854351 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec4a9e7-3f72-4b86-973d-a285b4d29f57 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:33:59.863387 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 37/70] 10.222.0.36 () {56 vars in 1101 bytes} [Wed Jul 27 10:33:59 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:33:59.864527 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137e11c5-af6e-40e2-a737-14fa1de3252d None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:33:59.864819 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137e11c5-af6e-40e2-a737-14fa1de3252d None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:33:59.865120 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137e11c5-af6e-40e2-a737-14fa1de3252d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:33:59.881966 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-137e11c5-af6e-40e2-a737-14fa1de3252d None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:33:59.883181 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-137e11c5-af6e-40e2-a737-14fa1de3252d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:33:59.916090 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:33:59.921516 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1567d31d-0cf7-4072-963f-68c463199138 None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:33:59.950962 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 38/71] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 10:33:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 10:33:59.951815 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137e11c5-af6e-40e2-a737-14fa1de3252d None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:33:59.956028 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a047d8-7b6f-43f9-8573-02b353d8ff78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:33:59.956747 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a047d8-7b6f-43f9-8573-02b353d8ff78 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:33:59.957104 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a047d8-7b6f-43f9-8573-02b353d8ff78 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:33:59.957449 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a047d8-7b6f-43f9-8573-02b353d8ff78 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:33:59.968474 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a047d8-7b6f-43f9-8573-02b353d8ff78 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 10:33:59.973784 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application [None req-94a047d8-7b6f-43f9-8573-02b353d8ff78 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 10:33:59.973784 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:33:59.973784 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:33:59.973784 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:33:59.973784 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:33:59.973784 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:33:59.973784 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:33:59.973784 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:33:59.973784 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:33:59.973784 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:33:59.973784 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:33:59.973784 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:33:59.973784 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 10:33:59.973784 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 10:33:59.973784 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 10:33:59.973784 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 10:33:59.974666 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 10:33:59.974666 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 10:33:59.974666 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 10:33:59.974666 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 10:33:59.974666 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:33:59.974666 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:33:59.974666 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:33:59.974666 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:33:59.974666 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:33:59.974666 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:33:59.974666 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:33:59.974666 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application with Lock( Jul 27 10:33:59.974666 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:33:59.974666 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:33:59.974666 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:33:59.974666 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:33:59.975431 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:33:59.975431 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:33:59.975431 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:33:59.975431 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:33:59.975431 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 10:33:59.975431 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 10:33:59.975431 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 10:33:59.975431 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 10:33:59.975431 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 10:33:59.975431 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 10:33:59.975431 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 10:33:59.975431 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Jul 27 10:33:59.975431 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 34/72] 10.222.0.36 () {56 vars in 1096 bytes} [Wed Jul 27 10:33:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 10:33:59.976647 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ab62fc-be95-489c-a4b2-55ea977deb57 None None] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:33:59.976932 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ab62fc-be95-489c-a4b2-55ea977deb57 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:33:59.977244 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ab62fc-be95-489c-a4b2-55ea977deb57 None None] PATH_INFO: `/` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:33:59.978722 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 35/73] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 10:33:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 10:33:59.983549 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ab62fc-be95-489c-a4b2-55ea977deb57 None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:00.076582 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137e11c5-af6e-40e2-a737-14fa1de3252d None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:00.091383 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ab62fc-be95-489c-a4b2-55ea977deb57 None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:00.099471 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:00.111426 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868fe084-b521-4f3c-acd0-6318f93ad2d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:00.112069 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868fe084-b521-4f3c-acd0-6318f93ad2d0 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:00.112352 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868fe084-b521-4f3c-acd0-6318f93ad2d0 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:00.112626 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868fe084-b521-4f3c-acd0-6318f93ad2d0 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:00.122522 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868fe084-b521-4f3c-acd0-6318f93ad2d0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 10:34:00.124697 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe7271b-691c-4208-aa59-be6bd8be5633 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:00.125338 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe7271b-691c-4208-aa59-be6bd8be5633 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:00.125626 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe7271b-691c-4208-aa59-be6bd8be5633 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:00.125907 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe7271b-691c-4208-aa59-be6bd8be5633 None admin] PATH_INFO: `/v3/users/6832ba6fe7d34e438a50d7f4d5681d00` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:00.127493 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application [None req-868fe084-b521-4f3c-acd0-6318f93ad2d0 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 27 10:34:00.127493 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:34:00.127493 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:34:00.127493 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:34:00.127493 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:34:00.127493 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:34:00.127493 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:34:00.127493 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:34:00.127493 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:34:00.127493 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:34:00.127493 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:34:00.127493 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:34:00.127493 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 10:34:00.127493 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 10:34:00.127493 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 10:34:00.127493 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 10:34:00.128469 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 10:34:00.128469 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 10:34:00.128469 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 10:34:00.128469 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 10:34:00.128469 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:34:00.128469 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:34:00.128469 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:34:00.128469 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:34:00.128469 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:34:00.128469 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:34:00.128469 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:34:00.128469 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application with Lock( Jul 27 10:34:00.128469 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:34:00.128469 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:34:00.128469 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:34:00.128469 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:34:00.129383 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:34:00.129383 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:34:00.129383 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:34:00.129383 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:34:00.129383 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 10:34:00.129383 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 10:34:00.129383 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 10:34:00.129383 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 10:34:00.129383 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 10:34:00.129383 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 10:34:00.129383 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 27 10:34:00.129383 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Jul 27 10:34:00.129383 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 39/74] 10.222.0.36 () {56 vars in 1114 bytes} [Wed Jul 27 10:33:59 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 10:34:00.130179 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868fe084-b521-4f3c-acd0-6318f93ad2d0 None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:00.146193 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe7271b-691c-4208-aa59-be6bd8be5633 None admin] RBAC: Authorizing `identity:get_user(user_id=6832ba6fe7d34e438a50d7f4d5681d00)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:00.149000 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe7271b-691c-4208-aa59-be6bd8be5633 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:00.151319 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 36/75] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Jul 27 10:33:59 2022] GET /identity/v3/users/6832ba6fe7d34e438a50d7f4d5681d00 => generated 275 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:34:00.152075 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe7271b-691c-4208-aa59-be6bd8be5633 None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:00.168857 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:00.231680 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868fe084-b521-4f3c-acd0-6318f93ad2d0 None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:00.248716 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe7271b-691c-4208-aa59-be6bd8be5633 None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:00.265669 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11d5977-309c-41aa-8c57-e2803b347e2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:00.266312 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11d5977-309c-41aa-8c57-e2803b347e2e None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:00.266593 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11d5977-309c-41aa-8c57-e2803b347e2e None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:00.266891 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11d5977-309c-41aa-8c57-e2803b347e2e None admin] PATH_INFO: `/v3/roles/member` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:00.276791 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11d5977-309c-41aa-8c57-e2803b347e2e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 10:34:00.281710 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application [None req-b11d5977-309c-41aa-8c57-e2803b347e2e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 27 10:34:00.281710 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:34:00.281710 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:34:00.281710 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:34:00.281710 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:34:00.281710 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:34:00.281710 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:34:00.281710 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:34:00.281710 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:34:00.281710 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:34:00.281710 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:34:00.281710 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:34:00.281710 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 10:34:00.281710 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 10:34:00.281710 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 10:34:00.281710 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 10:34:00.282523 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 10:34:00.282523 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 10:34:00.282523 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 10:34:00.282523 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 10:34:00.282523 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:34:00.282523 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:34:00.282523 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:34:00.282523 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:34:00.282523 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:34:00.282523 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:34:00.282523 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:34:00.282523 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application with Lock( Jul 27 10:34:00.282523 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:34:00.282523 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:34:00.282523 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:34:00.282523 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:34:00.283324 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:34:00.283324 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:34:00.283324 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:34:00.283324 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:34:00.283324 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 10:34:00.283324 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 10:34:00.283324 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 10:34:00.283324 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 10:34:00.283324 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 10:34:00.283324 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 10:34:00.283324 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 27 10:34:00.283324 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Jul 27 10:34:00.283887 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7d79fd-373e-4475-bef3-cc3b10363079 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:00.284265 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 40/76] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jul 27 10:34:00 2022] GET /identity/v3/roles/member => generated 84 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 10:34:00.284333 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7d79fd-373e-4475-bef3-cc3b10363079 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:00.284333 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7d79fd-373e-4475-bef3-cc3b10363079 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:00.284333 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7d79fd-373e-4475-bef3-cc3b10363079 None admin] PATH_INFO: `/v3/roles` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:00.284508 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd28f9e-5256-4386-ab76-98119ea9cc36 None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:00.284583 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7d79fd-373e-4475-bef3-cc3b10363079 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86828) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:34:00.284647 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd28f9e-5256-4386-ab76-98119ea9cc36 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:00.284647 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd28f9e-5256-4386-ab76-98119ea9cc36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:00.284764 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7d79fd-373e-4475-bef3-cc3b10363079 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:00.286001 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7d79fd-373e-4475-bef3-cc3b10363079 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:00.298425 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 37/77] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jul 27 10:34:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:34:00.299265 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7d79fd-373e-4475-bef3-cc3b10363079 None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:00.301863 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-0dd28f9e-5256-4386-ab76-98119ea9cc36 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:34:00.303199 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-0dd28f9e-5256-4386-ab76-98119ea9cc36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:34:00.339463 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:00.374343 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 41/78] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 10:34:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 10:34:00.384478 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd28f9e-5256-4386-ab76-98119ea9cc36 None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:00.384834 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:00.408214 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7d79fd-373e-4475-bef3-cc3b10363079 None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:00.441844 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159ed3b9-361d-4989-a424-8f6601554dd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:00.442474 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159ed3b9-361d-4989-a424-8f6601554dd6 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:00.442794 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159ed3b9-361d-4989-a424-8f6601554dd6 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:00.443081 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159ed3b9-361d-4989-a424-8f6601554dd6 None admin] PATH_INFO: `/v3/roles` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:00.443810 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159ed3b9-361d-4989-a424-8f6601554dd6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86828) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:34:00.444126 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159ed3b9-361d-4989-a424-8f6601554dd6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:00.445521 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159ed3b9-361d-4989-a424-8f6601554dd6 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:00.454248 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 38/79] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Jul 27 10:34:00 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:34:00.455074 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159ed3b9-361d-4989-a424-8f6601554dd6 None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:00.486337 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd28f9e-5256-4386-ab76-98119ea9cc36 None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:00.520087 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b01cacb-c67f-4a13-9d55-30e275a5baab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:00.520707 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b01cacb-c67f-4a13-9d55-30e275a5baab None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:00.520992 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b01cacb-c67f-4a13-9d55-30e275a5baab None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:00.521262 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b01cacb-c67f-4a13-9d55-30e275a5baab None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:00.554055 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159ed3b9-361d-4989-a424-8f6601554dd6 None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:00.561860 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:00.587635 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a30b7e7-beaf-4c87-a11e-d8138a7f8b98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:00.588250 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a30b7e7-beaf-4c87-a11e-d8138a7f8b98 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:00.588539 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a30b7e7-beaf-4c87-a11e-d8138a7f8b98 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:00.588723 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b01cacb-c67f-4a13-9d55-30e275a5baab None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:00.588842 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a30b7e7-beaf-4c87-a11e-d8138a7f8b98 None admin] PATH_INFO: `/v3/roles` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:00.589571 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a30b7e7-beaf-4c87-a11e-d8138a7f8b98 None admin] RBAC: Adding query filter params (name=member) {{(pid=86828) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:34:00.589831 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a30b7e7-beaf-4c87-a11e-d8138a7f8b98 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:00.590194 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b01cacb-c67f-4a13-9d55-30e275a5baab None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:00.591248 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a30b7e7-beaf-4c87-a11e-d8138a7f8b98 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:00.599210 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 39/80] 10.222.0.36 () {56 vars in 1101 bytes} [Wed Jul 27 10:34:00 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:34:00.600002 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a30b7e7-beaf-4c87-a11e-d8138a7f8b98 None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:00.603332 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:00.690628 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 42/81] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jul 27 10:34:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 10:34:00.691642 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b01cacb-c67f-4a13-9d55-30e275a5baab None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:00.699864 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a30b7e7-beaf-4c87-a11e-d8138a7f8b98 None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:00.736322 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36879d5-6701-42c7-b258-c058fdfddc24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:00.736964 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36879d5-6701-42c7-b258-c058fdfddc24 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:00.737241 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36879d5-6701-42c7-b258-c058fdfddc24 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:00.737516 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36879d5-6701-42c7-b258-c058fdfddc24 None admin] PATH_INFO: `/v3/users/60b0ea1b90004781b58a0cbfd103ae8f` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:00.738909 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36879d5-6701-42c7-b258-c058fdfddc24 None admin] RBAC: Authorizing `identity:get_user(user_id=60b0ea1b90004781b58a0cbfd103ae8f)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:00.740383 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36879d5-6701-42c7-b258-c058fdfddc24 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:00.742151 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 40/82] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Jul 27 10:34:00 2022] GET /identity/v3/users/60b0ea1b90004781b58a0cbfd103ae8f => generated 247 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:34:00.743406 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36879d5-6701-42c7-b258-c058fdfddc24 None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:00.774265 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:00.804412 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b01cacb-c67f-4a13-9d55-30e275a5baab None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:00.814411 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:00.839320 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6288da-084a-4ffa-8d00-87a54ce02ca3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:00.840024 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6288da-084a-4ffa-8d00-87a54ce02ca3 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:00.840255 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6288da-084a-4ffa-8d00-87a54ce02ca3 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:00.840541 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6288da-084a-4ffa-8d00-87a54ce02ca3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:00.841699 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6288da-084a-4ffa-8d00-87a54ce02ca3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:00.843206 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6288da-084a-4ffa-8d00-87a54ce02ca3 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:00.844848 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 43/83] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 10:34:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:34:00.849710 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6288da-084a-4ffa-8d00-87a54ce02ca3 None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:00.849795 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36879d5-6701-42c7-b258-c058fdfddc24 None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:00.880256 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345cdead-6591-4aae-831a-9171d0356735 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:00.880911 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345cdead-6591-4aae-831a-9171d0356735 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:00.881202 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345cdead-6591-4aae-831a-9171d0356735 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:00.881486 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345cdead-6591-4aae-831a-9171d0356735 None admin] PATH_INFO: `/v3/users/6832ba6fe7d34e438a50d7f4d5681d00` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:00.882707 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345cdead-6591-4aae-831a-9171d0356735 None admin] RBAC: Authorizing `identity:get_user(user_id=6832ba6fe7d34e438a50d7f4d5681d00)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:00.884204 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345cdead-6591-4aae-831a-9171d0356735 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:00.886067 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 41/84] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Jul 27 10:34:00 2022] GET /identity/v3/users/6832ba6fe7d34e438a50d7f4d5681d00 => generated 275 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:34:00.886845 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345cdead-6591-4aae-831a-9171d0356735 None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:00.949882 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6288da-084a-4ffa-8d00-87a54ce02ca3 None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:00.984293 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2150626-f483-422b-a89f-fd5dc6466f5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:00.984945 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2150626-f483-422b-a89f-fd5dc6466f5c None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:00.985227 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2150626-f483-422b-a89f-fd5dc6466f5c None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:00.985510 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2150626-f483-422b-a89f-fd5dc6466f5c None admin] PATH_INFO: `/v3/users/6832ba6fe7d34e438a50d7f4d5681d00` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:00.986735 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2150626-f483-422b-a89f-fd5dc6466f5c None admin] RBAC: Authorizing `identity:get_user(user_id=6832ba6fe7d34e438a50d7f4d5681d00)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:00.988200 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2150626-f483-422b-a89f-fd5dc6466f5c None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:00.988730 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345cdead-6591-4aae-831a-9171d0356735 None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:00.989818 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 44/85] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Jul 27 10:34:00 2022] GET /identity/v3/users/6832ba6fe7d34e438a50d7f4d5681d00 => generated 275 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:34:00.990583 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2150626-f483-422b-a89f-fd5dc6466f5c None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:01.000842 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:01.024928 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae08a6e-307f-4624-b8c2-aa72ab07f859 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:01.025584 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae08a6e-307f-4624-b8c2-aa72ab07f859 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:01.025874 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae08a6e-307f-4624-b8c2-aa72ab07f859 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:01.026161 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae08a6e-307f-4624-b8c2-aa72ab07f859 None admin] PATH_INFO: `/v3/projects/33cf2502c8694374bd447d6304e658c2` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:01.027386 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae08a6e-307f-4624-b8c2-aa72ab07f859 None admin] RBAC: Authorizing `identity:get_project(project_id=33cf2502c8694374bd447d6304e658c2)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:01.028839 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae08a6e-307f-4624-b8c2-aa72ab07f859 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:01.030512 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 42/86] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Jul 27 10:34:00 2022] GET /identity/v3/projects/33cf2502c8694374bd447d6304e658c2 => generated 298 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:34:01.031300 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae08a6e-307f-4624-b8c2-aa72ab07f859 None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:01.041164 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:01.094390 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2150626-f483-422b-a89f-fd5dc6466f5c None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:01.127508 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771f9021-2f24-491f-908a-491305a84931 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:01.128134 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771f9021-2f24-491f-908a-491305a84931 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:01.128423 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771f9021-2f24-491f-908a-491305a84931 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:01.128705 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771f9021-2f24-491f-908a-491305a84931 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:01.129529 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae08a6e-307f-4624-b8c2-aa72ab07f859 None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:01.162653 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e209da7-8277-49cc-ab43-aa2fc4555785 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:01.163266 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e209da7-8277-49cc-ab43-aa2fc4555785 None admin] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:01.163550 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e209da7-8277-49cc-ab43-aa2fc4555785 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:01.163821 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e209da7-8277-49cc-ab43-aa2fc4555785 None admin] PATH_INFO: `/v3/users` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:01.164496 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e209da7-8277-49cc-ab43-aa2fc4555785 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:01.166481 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e209da7-8277-49cc-ab43-aa2fc4555785 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:01.187427 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 43/87] 10.222.0.36 () {60 vars in 1132 bytes} [Wed Jul 27 10:34:01 2022] POST /identity/v3/users => generated 289 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 10:34:01.188452 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e209da7-8277-49cc-ab43-aa2fc4555785 None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:01.200362 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771f9021-2f24-491f-908a-491305a84931 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:01.202293 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771f9021-2f24-491f-908a-491305a84931 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:01.214480 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:01.260178 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:01.306367 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e209da7-8277-49cc-ab43-aa2fc4555785 None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:01.321624 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 45/88] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jul 27 10:34:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 10:34:01.322511 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771f9021-2f24-491f-908a-491305a84931 None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:01.351718 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e0a798-2d4e-4e33-a624-977d187ccae9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:01.353259 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e0a798-2d4e-4e33-a624-977d187ccae9 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:01.354136 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e0a798-2d4e-4e33-a624-977d187ccae9 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:01.355107 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e0a798-2d4e-4e33-a624-977d187ccae9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:01.436673 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e0a798-2d4e-4e33-a624-977d187ccae9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:01.438169 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e0a798-2d4e-4e33-a624-977d187ccae9 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:01.440225 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771f9021-2f24-491f-908a-491305a84931 None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:01.452982 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:01.491704 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034772c8-c059-47b5-9c65-67f4005fc896 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:01.492707 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034772c8-c059-47b5-9c65-67f4005fc896 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:01.493000 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034772c8-c059-47b5-9c65-67f4005fc896 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:01.493570 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034772c8-c059-47b5-9c65-67f4005fc896 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:01.515871 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:01.563360 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 44/89] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jul 27 10:34:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 10:34:01.564554 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e0a798-2d4e-4e33-a624-977d187ccae9 None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:01.583127 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034772c8-c059-47b5-9c65-67f4005fc896 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:01.584749 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034772c8-c059-47b5-9c65-67f4005fc896 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:01.677805 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e0a798-2d4e-4e33-a624-977d187ccae9 None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:01.700754 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 46/90] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jul 27 10:34:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 10:34:01.700754 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034772c8-c059-47b5-9c65-67f4005fc896 None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:01.708258 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:01.723409 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3f7ed7-0c9d-4224-a7d7-22a77d02107d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:01.724504 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3f7ed7-0c9d-4224-a7d7-22a77d02107d None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:01.725041 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3f7ed7-0c9d-4224-a7d7-22a77d02107d None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:01.725520 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3f7ed7-0c9d-4224-a7d7-22a77d02107d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:01.726533 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3f7ed7-0c9d-4224-a7d7-22a77d02107d None admin] RBAC: Adding query filter params (role.id=4c9fc0b3baed441da2126be41cc6b04a, scope.project.id=33cf2502c8694374bd447d6304e658c2, user.id=6832ba6fe7d34e438a50d7f4d5681d00) {{(pid=86828) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:34:01.727217 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3f7ed7-0c9d-4224-a7d7-22a77d02107d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:01.728993 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3f7ed7-0c9d-4224-a7d7-22a77d02107d None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:01.737484 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 45/91] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Jul 27 10:34:01 2022] GET /identity/v3/role_assignments?user.id=6832ba6fe7d34e438a50d7f4d5681d00&scope.project.id=33cf2502c8694374bd447d6304e658c2&role.id=4c9fc0b3baed441da2126be41cc6b04a => generated 259 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:34:01.738645 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3f7ed7-0c9d-4224-a7d7-22a77d02107d None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:01.771385 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:01.832729 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034772c8-c059-47b5-9c65-67f4005fc896 None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:01.874252 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60395b14-e069-4b8e-8350-c16119b7ba20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:01.875111 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60395b14-e069-4b8e-8350-c16119b7ba20 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:01.875399 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60395b14-e069-4b8e-8350-c16119b7ba20 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:01.875685 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60395b14-e069-4b8e-8350-c16119b7ba20 None admin] PATH_INFO: `/v3/projects/33cf2502c8694374bd447d6304e658c2` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:01.877521 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60395b14-e069-4b8e-8350-c16119b7ba20 None admin] RBAC: Authorizing `identity:get_project(project_id=33cf2502c8694374bd447d6304e658c2)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:01.879058 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60395b14-e069-4b8e-8350-c16119b7ba20 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:01.880814 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 47/92] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Jul 27 10:34:01 2022] GET /identity/v3/projects/33cf2502c8694374bd447d6304e658c2 => generated 298 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:34:01.881604 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60395b14-e069-4b8e-8350-c16119b7ba20 None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:01.931298 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3f7ed7-0c9d-4224-a7d7-22a77d02107d None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:01.981703 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:02.010269 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8075ff-8b2f-4236-897b-af46dbbf28ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:02.011311 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8075ff-8b2f-4236-897b-af46dbbf28ef None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:02.012376 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8075ff-8b2f-4236-897b-af46dbbf28ef None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:02.012987 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8075ff-8b2f-4236-897b-af46dbbf28ef None admin] PATH_INFO: `/v3/projects/33cf2502c8694374bd447d6304e658c2` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:02.016180 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8075ff-8b2f-4236-897b-af46dbbf28ef None admin] RBAC: Authorizing `identity:get_project(project_id=33cf2502c8694374bd447d6304e658c2)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:02.018402 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8075ff-8b2f-4236-897b-af46dbbf28ef None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:02.020434 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 46/93] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Jul 27 10:34:01 2022] GET /identity/v3/projects/33cf2502c8694374bd447d6304e658c2 => generated 298 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:34:02.022513 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8075ff-8b2f-4236-897b-af46dbbf28ef None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:02.030148 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60395b14-e069-4b8e-8350-c16119b7ba20 None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:02.118128 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5124966-bc71-44fe-adb1-031f2f4f5169 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:02.119059 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5124966-bc71-44fe-adb1-031f2f4f5169 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:02.119492 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5124966-bc71-44fe-adb1-031f2f4f5169 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:02.119917 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5124966-bc71-44fe-adb1-031f2f4f5169 None admin] PATH_INFO: `/v3/projects/cb7449fbc55341b2a1a7bb436dcd7b35` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:02.122047 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5124966-bc71-44fe-adb1-031f2f4f5169 None admin] RBAC: Authorizing `identity:get_project(project_id=cb7449fbc55341b2a1a7bb436dcd7b35)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:02.123688 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5124966-bc71-44fe-adb1-031f2f4f5169 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:02.129702 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 48/94] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Jul 27 10:34:01 2022] GET /identity/v3/projects/cb7449fbc55341b2a1a7bb436dcd7b35 => generated 312 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:34:02.130703 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5124966-bc71-44fe-adb1-031f2f4f5169 None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:02.225017 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:02.238306 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8075ff-8b2f-4236-897b-af46dbbf28ef None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:02.281840 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5124966-bc71-44fe-adb1-031f2f4f5169 None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:02.318148 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9ac802-7dfe-40ca-b07c-66ef3dfffd74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:02.319828 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9ac802-7dfe-40ca-b07c-66ef3dfffd74 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:02.320675 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9ac802-7dfe-40ca-b07c-66ef3dfffd74 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:02.321957 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9ac802-7dfe-40ca-b07c-66ef3dfffd74 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:02.323452 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9ac802-7dfe-40ca-b07c-66ef3dfffd74 None admin] RBAC: Adding query filter params (role.id=21156560eb6b436bb567a40b4976f3f4, scope.project.id=33cf2502c8694374bd447d6304e658c2, user.id=60b0ea1b90004781b58a0cbfd103ae8f) {{(pid=86828) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:34:02.324201 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9da21f-6b52-4c29-8657-4768bbd6c88f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:02.325263 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9da21f-6b52-4c29-8657-4768bbd6c88f None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:02.325728 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9ac802-7dfe-40ca-b07c-66ef3dfffd74 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:02.326173 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9da21f-6b52-4c29-8657-4768bbd6c88f None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:02.327794 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9ac802-7dfe-40ca-b07c-66ef3dfffd74 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:02.328239 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9da21f-6b52-4c29-8657-4768bbd6c88f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:02.329184 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9da21f-6b52-4c29-8657-4768bbd6c88f None admin] RBAC: Adding query filter params (role.id=c3341f95596344309327efff99a20d5c, scope.project.id=33cf2502c8694374bd447d6304e658c2, user.id=6832ba6fe7d34e438a50d7f4d5681d00) {{(pid=86827) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:34:02.329654 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9da21f-6b52-4c29-8657-4768bbd6c88f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:02.333284 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9da21f-6b52-4c29-8657-4768bbd6c88f None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:02.340650 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 49/95] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Jul 27 10:34:02 2022] GET /identity/v3/role_assignments?user.id=6832ba6fe7d34e438a50d7f4d5681d00&scope.project.id=33cf2502c8694374bd447d6304e658c2&role.id=c3341f95596344309327efff99a20d5c => generated 259 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:34:02.341732 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9da21f-6b52-4c29-8657-4768bbd6c88f None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:02.344091 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 47/96] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Jul 27 10:34:02 2022] GET /identity/v3/role_assignments?user.id=60b0ea1b90004781b58a0cbfd103ae8f&scope.project.id=33cf2502c8694374bd447d6304e658c2&role.id=21156560eb6b436bb567a40b4976f3f4 => generated 259 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:34:02.355950 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:02.460859 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9da21f-6b52-4c29-8657-4768bbd6c88f None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:02.500327 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76ae886-51df-4141-9f26-d8c2f7cc474c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:02.501221 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76ae886-51df-4141-9f26-d8c2f7cc474c None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:02.501662 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76ae886-51df-4141-9f26-d8c2f7cc474c None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:02.502091 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76ae886-51df-4141-9f26-d8c2f7cc474c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:02.503159 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76ae886-51df-4141-9f26-d8c2f7cc474c None admin] RBAC: Adding query filter params (role.id=4c9fc0b3baed441da2126be41cc6b04a, scope.project.id=cb7449fbc55341b2a1a7bb436dcd7b35, user.id=6832ba6fe7d34e438a50d7f4d5681d00) {{(pid=86827) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:34:02.503632 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76ae886-51df-4141-9f26-d8c2f7cc474c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:02.505278 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76ae886-51df-4141-9f26-d8c2f7cc474c None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:02.512639 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 50/97] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Jul 27 10:34:02 2022] GET /identity/v3/role_assignments?user.id=6832ba6fe7d34e438a50d7f4d5681d00&scope.project.id=cb7449fbc55341b2a1a7bb436dcd7b35&role.id=4c9fc0b3baed441da2126be41cc6b04a => generated 259 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:34:03.814107 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da47d9d6-fce5-41a9-989b-39a0b28d050a None None] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:03.814649 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da47d9d6-fce5-41a9-989b-39a0b28d050a None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:03.814728 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da47d9d6-fce5-41a9-989b-39a0b28d050a None None] PATH_INFO: `/` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:03.816221 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 48/98] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 10:34:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 10:34:03.824807 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e89423-28d0-4574-91a7-72e86e56682b None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:03.824807 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e89423-28d0-4574-91a7-72e86e56682b None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:03.824807 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e89423-28d0-4574-91a7-72e86e56682b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:03.851625 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-13e89423-28d0-4574-91a7-72e86e56682b None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:34:03.853671 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-13e89423-28d0-4574-91a7-72e86e56682b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:34:03.960051 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 51/99] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 10:34:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 10:34:03.995821 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da47d9d6-fce5-41a9-989b-39a0b28d050a None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:04.098384 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f3c7a1-9791-46b0-b3f1-cf878655d06c None None] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:04.098690 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f3c7a1-9791-46b0-b3f1-cf878655d06c None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:04.098898 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f3c7a1-9791-46b0-b3f1-cf878655d06c None None] PATH_INFO: `/` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:04.100405 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 52/100] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 10:34:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 10:34:04.111270 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367c3e29-462b-490d-bec4-4f1bb803d853 None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:04.111730 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367c3e29-462b-490d-bec4-4f1bb803d853 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:04.112179 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367c3e29-462b-490d-bec4-4f1bb803d853 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:04.130528 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:04.131614 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da47d9d6-fce5-41a9-989b-39a0b28d050a None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:04.158807 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-367c3e29-462b-490d-bec4-4f1bb803d853 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:34:04.160381 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-367c3e29-462b-490d-bec4-4f1bb803d853 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:34:04.200149 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02763985-e39f-40c6-8b3e-f9cbede587d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:04.201323 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02763985-e39f-40c6-8b3e-f9cbede587d5 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:04.202089 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02763985-e39f-40c6-8b3e-f9cbede587d5 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:04.202951 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02763985-e39f-40c6-8b3e-f9cbede587d5 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:04.219052 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02763985-e39f-40c6-8b3e-f9cbede587d5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 10:34:04.226007 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application [None req-02763985-e39f-40c6-8b3e-f9cbede587d5 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 27 10:34:04.226007 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:34:04.226007 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:34:04.226007 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:34:04.226007 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:34:04.226007 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:34:04.226007 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:34:04.226007 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:34:04.226007 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:34:04.226007 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:34:04.226007 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:34:04.226007 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:34:04.226007 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 10:34:04.226007 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 10:34:04.226007 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 10:34:04.226007 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 10:34:04.227034 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 10:34:04.227034 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 10:34:04.227034 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 10:34:04.227034 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 10:34:04.227034 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:34:04.227034 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:34:04.227034 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:34:04.227034 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:34:04.227034 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:34:04.227034 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:34:04.227034 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:34:04.227034 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application with Lock( Jul 27 10:34:04.227034 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:34:04.227034 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:34:04.227034 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:34:04.227034 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:34:04.227781 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:34:04.227781 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:34:04.227781 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:34:04.227781 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:34:04.227781 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 10:34:04.227781 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 10:34:04.227781 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 10:34:04.227781 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 10:34:04.227781 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 10:34:04.227781 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 10:34:04.227781 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 27 10:34:04.227781 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Jul 27 10:34:04.230381 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 49/101] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jul 27 10:34:03 2022] GET /identity/v3/roles/reader => generated 84 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 10:34:04.236035 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02763985-e39f-40c6-8b3e-f9cbede587d5 None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:04.265969 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:04.371048 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02763985-e39f-40c6-8b3e-f9cbede587d5 None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:04.404515 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 53/102] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 10:34:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 295 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 10:34:04.407696 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b378e866-9166-4fc0-8565-0ac6d1cb190c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:04.408579 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b378e866-9166-4fc0-8565-0ac6d1cb190c None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:04.409067 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b378e866-9166-4fc0-8565-0ac6d1cb190c None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:04.409546 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b378e866-9166-4fc0-8565-0ac6d1cb190c None admin] PATH_INFO: `/v3/roles` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:04.410502 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b378e866-9166-4fc0-8565-0ac6d1cb190c None admin] RBAC: Adding query filter params (name=reader) {{(pid=86828) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:34:04.411040 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b378e866-9166-4fc0-8565-0ac6d1cb190c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:04.412713 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b378e866-9166-4fc0-8565-0ac6d1cb190c None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:04.422294 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 50/103] 10.222.0.36 () {56 vars in 1101 bytes} [Wed Jul 27 10:34:04 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 10:34:04.427772 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367c3e29-462b-490d-bec4-4f1bb803d853 None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:04.448332 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b378e866-9166-4fc0-8565-0ac6d1cb190c None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:04.568043 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:04.630209 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b378e866-9166-4fc0-8565-0ac6d1cb190c None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:04.646295 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367c3e29-462b-490d-bec4-4f1bb803d853 None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:04.684388 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ac8d38-301b-47d0-ad74-6e37f3f7e4b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:04.685458 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ac8d38-301b-47d0-ad74-6e37f3f7e4b2 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:04.685876 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ac8d38-301b-47d0-ad74-6e37f3f7e4b2 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:04.686286 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ac8d38-301b-47d0-ad74-6e37f3f7e4b2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:04.688154 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ac8d38-301b-47d0-ad74-6e37f3f7e4b2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:04.689770 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ac8d38-301b-47d0-ad74-6e37f3f7e4b2 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:04.692064 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 51/104] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 10:34:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 10:34:04.697583 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ac8d38-301b-47d0-ad74-6e37f3f7e4b2 None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:04.709922 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:04.720975 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e54ed3-de9a-4ae0-a505-d599f0fc54ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:04.721853 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e54ed3-de9a-4ae0-a505-d599f0fc54ea None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:04.723564 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e54ed3-de9a-4ae0-a505-d599f0fc54ea None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:04.723993 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e54ed3-de9a-4ae0-a505-d599f0fc54ea None admin] PATH_INFO: `/v3/users/aa21fc3e363e487d94f11e274b824cf8` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:04.743082 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e54ed3-de9a-4ae0-a505-d599f0fc54ea None admin] RBAC: Authorizing `identity:get_user(user_id=aa21fc3e363e487d94f11e274b824cf8)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:04.744939 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e54ed3-de9a-4ae0-a505-d599f0fc54ea None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:04.747396 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 54/105] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Jul 27 10:34:04 2022] GET /identity/v3/users/aa21fc3e363e487d94f11e274b824cf8 => generated 289 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:34:04.748548 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320e1132-7ffc-45dc-840a-22a55c1050c0 None None] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:04.748947 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320e1132-7ffc-45dc-840a-22a55c1050c0 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:04.749342 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320e1132-7ffc-45dc-840a-22a55c1050c0 None None] PATH_INFO: `/` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:04.751406 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 55/106] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 10:34:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 10:34:04.756364 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320e1132-7ffc-45dc-840a-22a55c1050c0 None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:04.877852 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320e1132-7ffc-45dc-840a-22a55c1050c0 None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:04.918022 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255b7105-25e0-46b4-a63a-dbfc4317e313 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:04.918720 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255b7105-25e0-46b4-a63a-dbfc4317e313 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:04.919082 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255b7105-25e0-46b4-a63a-dbfc4317e313 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:04.919437 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255b7105-25e0-46b4-a63a-dbfc4317e313 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:04.935320 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ac8d38-301b-47d0-ad74-6e37f3f7e4b2 None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:04.993024 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:05.004264 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255b7105-25e0-46b4-a63a-dbfc4317e313 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:05.005979 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255b7105-25e0-46b4-a63a-dbfc4317e313 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:05.015315 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60cd215-1e1b-4e1d-82fd-022e526eebef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:05.015970 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60cd215-1e1b-4e1d-82fd-022e526eebef None admin] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:05.016261 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60cd215-1e1b-4e1d-82fd-022e526eebef None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:05.016542 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60cd215-1e1b-4e1d-82fd-022e526eebef None admin] PATH_INFO: `/v3/projects` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:05.017238 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60cd215-1e1b-4e1d-82fd-022e526eebef None admin] RBAC: Authorizing `identity:create_project()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:05.018882 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60cd215-1e1b-4e1d-82fd-022e526eebef None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:05.067848 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:05.080872 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 52/107] 10.222.0.36 () {60 vars in 1141 bytes} [Wed Jul 27 10:34:04 2022] POST /identity/v3/projects => generated 302 bytes in 383 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 10:34:05.082168 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb38497-0804-4471-9d6b-bf0f48cf1cf2 None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:05.087158 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb38497-0804-4471-9d6b-bf0f48cf1cf2 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:05.087507 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb38497-0804-4471-9d6b-bf0f48cf1cf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:05.118118 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 56/108] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jul 27 10:34:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 10:34:05.123645 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-3cb38497-0804-4471-9d6b-bf0f48cf1cf2 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:34:05.125042 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-3cb38497-0804-4471-9d6b-bf0f48cf1cf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:34:05.128030 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255b7105-25e0-46b4-a63a-dbfc4317e313 None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:05.234968 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 53/109] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 10:34:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 10:34:05.279260 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:05.284670 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255b7105-25e0-46b4-a63a-dbfc4317e313 None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:05.299157 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfe552a-b640-4ca6-ad1b-5ebfa49c20a7 None None] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:05.299460 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfe552a-b640-4ca6-ad1b-5ebfa49c20a7 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:05.299725 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfe552a-b640-4ca6-ad1b-5ebfa49c20a7 None None] PATH_INFO: `/` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:05.301127 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 54/110] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 10:34:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 10:34:05.301939 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfe552a-b640-4ca6-ad1b-5ebfa49c20a7 None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:05.343328 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be43185-7ddc-42c1-9429-8e3bb2ad7801 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:05.344344 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be43185-7ddc-42c1-9429-8e3bb2ad7801 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:05.345100 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be43185-7ddc-42c1-9429-8e3bb2ad7801 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:05.346499 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be43185-7ddc-42c1-9429-8e3bb2ad7801 None admin] PATH_INFO: `/v3/projects/33cf2502c8694374bd447d6304e658c2` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:05.348199 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be43185-7ddc-42c1-9429-8e3bb2ad7801 None admin] RBAC: Authorizing `identity:get_project(project_id=33cf2502c8694374bd447d6304e658c2)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:05.349860 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be43185-7ddc-42c1-9429-8e3bb2ad7801 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:05.353294 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 57/111] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Jul 27 10:34:05 2022] GET /identity/v3/projects/33cf2502c8694374bd447d6304e658c2 => generated 298 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 10:34:05.355297 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bdb183-1cbb-4eaf-8465-3962b61bbea6 None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:05.355789 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bdb183-1cbb-4eaf-8465-3962b61bbea6 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:05.356771 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bdb183-1cbb-4eaf-8465-3962b61bbea6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:05.380374 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-f4bdb183-1cbb-4eaf-8465-3962b61bbea6 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:34:05.382359 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-f4bdb183-1cbb-4eaf-8465-3962b61bbea6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:34:05.434073 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfe552a-b640-4ca6-ad1b-5ebfa49c20a7 None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:05.467232 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:05.474296 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c7ed29-1c4e-4e22-82d3-86422a76881e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:05.474992 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c7ed29-1c4e-4e22-82d3-86422a76881e None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:05.475283 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c7ed29-1c4e-4e22-82d3-86422a76881e None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:05.475562 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c7ed29-1c4e-4e22-82d3-86422a76881e None admin] PATH_INFO: `/v3/roles/member` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:05.482408 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 58/112] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 10:34:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 10:34:05.483923 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bdb183-1cbb-4eaf-8465-3962b61bbea6 None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:05.488264 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c7ed29-1c4e-4e22-82d3-86422a76881e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 10:34:05.493175 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application [None req-96c7ed29-1c4e-4e22-82d3-86422a76881e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 27 10:34:05.493175 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:34:05.493175 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:34:05.493175 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:34:05.493175 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:34:05.493175 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:34:05.493175 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:34:05.493175 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:34:05.493175 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:34:05.493175 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:34:05.493175 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:34:05.493175 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:34:05.493175 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 10:34:05.493175 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 10:34:05.493175 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 10:34:05.493175 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 10:34:05.493996 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 10:34:05.493996 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 10:34:05.493996 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 10:34:05.493996 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 10:34:05.493996 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:34:05.493996 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:34:05.493996 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:34:05.493996 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:34:05.493996 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:34:05.493996 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:34:05.493996 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:34:05.493996 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application with Lock( Jul 27 10:34:05.493996 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:34:05.493996 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:34:05.493996 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:34:05.493996 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:34:05.495165 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:34:05.495165 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:34:05.495165 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:34:05.495165 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:34:05.495165 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 10:34:05.495165 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 10:34:05.495165 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 10:34:05.495165 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 10:34:05.495165 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 10:34:05.495165 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 10:34:05.495165 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 27 10:34:05.495165 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Jul 27 10:34:05.495165 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 55/113] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jul 27 10:34:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 10:34:05.500059 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c7ed29-1c4e-4e22-82d3-86422a76881e None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:05.611144 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:05.627784 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bdb183-1cbb-4eaf-8465-3962b61bbea6 None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:05.644744 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c7ed29-1c4e-4e22-82d3-86422a76881e None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:05.669096 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac77e99-12f7-46de-a093-b4ad58a2602e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:05.670102 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac77e99-12f7-46de-a093-b4ad58a2602e None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:05.670576 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac77e99-12f7-46de-a093-b4ad58a2602e None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:05.671088 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac77e99-12f7-46de-a093-b4ad58a2602e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:05.672372 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac77e99-12f7-46de-a093-b4ad58a2602e None admin] RBAC: Adding query filter params (role.id=80f67bfde4f5494e9c6d825aafb44e11, scope.project.id=33cf2502c8694374bd447d6304e658c2, user.id=aa21fc3e363e487d94f11e274b824cf8) {{(pid=86827) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:34:05.672908 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac77e99-12f7-46de-a093-b4ad58a2602e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:05.674825 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac77e99-12f7-46de-a093-b4ad58a2602e None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:05.681712 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 59/114] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Jul 27 10:34:05 2022] GET /identity/v3/role_assignments?user.id=aa21fc3e363e487d94f11e274b824cf8&scope.project.id=33cf2502c8694374bd447d6304e658c2&role.id=80f67bfde4f5494e9c6d825aafb44e11 => generated 259 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:34:05.683094 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac77e99-12f7-46de-a093-b4ad58a2602e None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:05.739435 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f61431d-aa8d-4750-8c6c-5192910d6bab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:05.740006 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f61431d-aa8d-4750-8c6c-5192910d6bab None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:05.740297 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f61431d-aa8d-4750-8c6c-5192910d6bab None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:05.740578 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f61431d-aa8d-4750-8c6c-5192910d6bab None admin] PATH_INFO: `/v3/roles` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:05.741321 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f61431d-aa8d-4750-8c6c-5192910d6bab None admin] RBAC: Adding query filter params (name=member) {{(pid=86828) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:34:05.741649 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f61431d-aa8d-4750-8c6c-5192910d6bab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:05.743636 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f61431d-aa8d-4750-8c6c-5192910d6bab None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:05.756813 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 56/115] 10.222.0.36 () {56 vars in 1101 bytes} [Wed Jul 27 10:34:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 10:34:05.758115 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc9c96c-c353-4404-9cb1-6de100d5110e None None] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:05.758961 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc9c96c-c353-4404-9cb1-6de100d5110e None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:05.759238 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc9c96c-c353-4404-9cb1-6de100d5110e None None] PATH_INFO: `/` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:05.760978 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 57/116] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 10:34:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 10:34:05.767523 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc9c96c-c353-4404-9cb1-6de100d5110e None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:05.846469 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac77e99-12f7-46de-a093-b4ad58a2602e None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:05.889354 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:05.926655 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4e69c2-5235-46f5-b90f-2357bef255f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:05.926655 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4e69c2-5235-46f5-b90f-2357bef255f4 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:05.926655 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4e69c2-5235-46f5-b90f-2357bef255f4 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:05.926655 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4e69c2-5235-46f5-b90f-2357bef255f4 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:05.934967 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4e69c2-5235-46f5-b90f-2357bef255f4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 10:34:05.943801 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application [None req-1e4e69c2-5235-46f5-b90f-2357bef255f4 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 27 10:34:05.943801 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:34:05.943801 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:34:05.943801 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:34:05.943801 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:34:05.943801 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:34:05.943801 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:34:05.943801 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:34:05.943801 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:34:05.943801 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:34:05.943801 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:34:05.943801 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:34:05.943801 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 10:34:05.943801 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 10:34:05.943801 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 10:34:05.943801 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 10:34:05.944648 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 10:34:05.944648 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 10:34:05.944648 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 10:34:05.944648 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 10:34:05.944648 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:34:05.944648 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:34:05.944648 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:34:05.944648 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:34:05.944648 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:34:05.944648 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:34:05.944648 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:34:05.944648 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application with Lock( Jul 27 10:34:05.944648 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:34:05.944648 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:34:05.944648 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:34:05.944648 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:34:05.945488 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:34:05.945488 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:34:05.945488 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:34:05.945488 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:34:05.945488 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 10:34:05.945488 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 10:34:05.945488 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 10:34:05.945488 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 10:34:05.945488 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 10:34:05.945488 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 10:34:05.945488 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 27 10:34:05.945488 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Jul 27 10:34:05.945488 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 60/117] 10.222.0.36 () {56 vars in 1114 bytes} [Wed Jul 27 10:34:05 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 10:34:05.946530 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37ea317-bce8-4dfe-848e-57ddbd30ae60 None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:05.946530 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37ea317-bce8-4dfe-848e-57ddbd30ae60 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:05.946822 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37ea317-bce8-4dfe-848e-57ddbd30ae60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:05.960538 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc9c96c-c353-4404-9cb1-6de100d5110e None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:05.969937 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-f37ea317-bce8-4dfe-848e-57ddbd30ae60 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:34:05.971187 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-f37ea317-bce8-4dfe-848e-57ddbd30ae60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:34:05.979591 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:05.997380 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f49cac5-49a1-457d-a377-962292afe295 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:05.998064 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f49cac5-49a1-457d-a377-962292afe295 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:05.998358 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f49cac5-49a1-457d-a377-962292afe295 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:05.998775 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f49cac5-49a1-457d-a377-962292afe295 None admin] PATH_INFO: `/v3/users/6832ba6fe7d34e438a50d7f4d5681d00` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:05.999988 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f49cac5-49a1-457d-a377-962292afe295 None admin] RBAC: Authorizing `identity:get_user(user_id=6832ba6fe7d34e438a50d7f4d5681d00)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:06.001730 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f49cac5-49a1-457d-a377-962292afe295 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:06.003451 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 58/118] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Jul 27 10:34:05 2022] GET /identity/v3/users/6832ba6fe7d34e438a50d7f4d5681d00 => generated 275 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:34:06.004809 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81c3540-427d-4e97-b92f-4a2462dafe83 None None] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:06.005103 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81c3540-427d-4e97-b92f-4a2462dafe83 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:06.005386 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81c3540-427d-4e97-b92f-4a2462dafe83 None None] PATH_INFO: `/` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:06.011302 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 59/119] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 10:34:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 10:34:06.012332 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81c3540-427d-4e97-b92f-4a2462dafe83 None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:06.054724 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 61/120] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 10:34:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 10:34:06.056297 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37ea317-bce8-4dfe-848e-57ddbd30ae60 None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:06.126212 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81c3540-427d-4e97-b92f-4a2462dafe83 None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:06.150977 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:06.167257 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32aafb7a-680e-4115-b1f6-bf24aca7ad61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:06.168656 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32aafb7a-680e-4115-b1f6-bf24aca7ad61 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:06.169394 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32aafb7a-680e-4115-b1f6-bf24aca7ad61 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:06.169684 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32aafb7a-680e-4115-b1f6-bf24aca7ad61 None admin] PATH_INFO: `/v3/roles` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:06.170704 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32aafb7a-680e-4115-b1f6-bf24aca7ad61 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86828) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:34:06.171029 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32aafb7a-680e-4115-b1f6-bf24aca7ad61 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:06.173400 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32aafb7a-680e-4115-b1f6-bf24aca7ad61 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:06.181440 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37ea317-bce8-4dfe-848e-57ddbd30ae60 None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:06.188577 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 60/121] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Jul 27 10:34:06 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:34:06.189714 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18fefc2-e5ea-47de-a062-cb9c6154403b None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:06.190050 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18fefc2-e5ea-47de-a062-cb9c6154403b None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:06.190385 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18fefc2-e5ea-47de-a062-cb9c6154403b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:06.214080 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-e18fefc2-e5ea-47de-a062-cb9c6154403b None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:34:06.216042 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-e18fefc2-e5ea-47de-a062-cb9c6154403b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:34:06.226795 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88e68fe-3c52-40f6-9624-4e80a0065e8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:06.227997 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88e68fe-3c52-40f6-9624-4e80a0065e8d None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:06.228282 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88e68fe-3c52-40f6-9624-4e80a0065e8d None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:06.228559 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88e68fe-3c52-40f6-9624-4e80a0065e8d None admin] PATH_INFO: `/v3/projects/33cf2502c8694374bd447d6304e658c2` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:06.229890 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88e68fe-3c52-40f6-9624-4e80a0065e8d None admin] RBAC: Authorizing `identity:get_project(project_id=33cf2502c8694374bd447d6304e658c2)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:06.231385 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88e68fe-3c52-40f6-9624-4e80a0065e8d None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:06.233227 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 62/122] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Jul 27 10:34:06 2022] GET /identity/v3/projects/33cf2502c8694374bd447d6304e658c2 => generated 298 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:34:06.234075 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88e68fe-3c52-40f6-9624-4e80a0065e8d None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:06.247326 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:06.313894 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 61/123] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 10:34:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 10:34:06.319460 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18fefc2-e5ea-47de-a062-cb9c6154403b None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:06.369750 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88e68fe-3c52-40f6-9624-4e80a0065e8d None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:06.420089 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c8ba54-b5c2-4c44-baf9-824b685b41f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:06.420736 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c8ba54-b5c2-4c44-baf9-824b685b41f0 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:06.421017 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c8ba54-b5c2-4c44-baf9-824b685b41f0 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:06.421290 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c8ba54-b5c2-4c44-baf9-824b685b41f0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:06.437185 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c8ba54-b5c2-4c44-baf9-824b685b41f0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 10:34:06.445095 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application [None req-a5c8ba54-b5c2-4c44-baf9-824b685b41f0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 10:34:06.445095 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:34:06.445095 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:34:06.445095 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:34:06.445095 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:34:06.445095 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:34:06.445095 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:34:06.445095 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:34:06.445095 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:34:06.445095 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:34:06.445095 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:34:06.445095 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:34:06.445095 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 10:34:06.445095 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 10:34:06.445095 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 10:34:06.445095 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 10:34:06.447256 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 10:34:06.447256 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 10:34:06.447256 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 10:34:06.447256 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 10:34:06.447256 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:34:06.447256 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:34:06.447256 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:34:06.447256 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:34:06.447256 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:34:06.447256 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:34:06.447256 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:34:06.447256 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application with Lock( Jul 27 10:34:06.447256 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:34:06.447256 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:34:06.447256 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:34:06.447256 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:34:06.449165 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:34:06.449165 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:34:06.449165 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:34:06.449165 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:34:06.449165 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 10:34:06.449165 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 10:34:06.449165 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 10:34:06.449165 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 10:34:06.449165 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 10:34:06.449165 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 10:34:06.449165 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 10:34:06.449165 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Jul 27 10:34:06.449165 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 63/124] 10.222.0.36 () {56 vars in 1096 bytes} [Wed Jul 27 10:34:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 10:34:06.449165 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c8ba54-b5c2-4c44-baf9-824b685b41f0 None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:06.450291 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18fefc2-e5ea-47de-a062-cb9c6154403b None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:06.463719 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:06.488862 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765dcf98-247b-437e-a3ab-1eb64bddd3b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:06.489707 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765dcf98-247b-437e-a3ab-1eb64bddd3b5 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:06.490087 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765dcf98-247b-437e-a3ab-1eb64bddd3b5 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:06.490494 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765dcf98-247b-437e-a3ab-1eb64bddd3b5 None admin] PATH_INFO: `/v3/users/6832ba6fe7d34e438a50d7f4d5681d00` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:06.491802 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765dcf98-247b-437e-a3ab-1eb64bddd3b5 None admin] RBAC: Authorizing `identity:get_user(user_id=6832ba6fe7d34e438a50d7f4d5681d00)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:06.493903 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765dcf98-247b-437e-a3ab-1eb64bddd3b5 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:06.495944 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 62/125] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Jul 27 10:34:06 2022] GET /identity/v3/users/6832ba6fe7d34e438a50d7f4d5681d00 => generated 275 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:34:06.496759 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765dcf98-247b-437e-a3ab-1eb64bddd3b5 None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:06.539651 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:06.573625 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c8ba54-b5c2-4c44-baf9-824b685b41f0 None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:06.610708 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a005530a-997d-45af-9103-3076e22a9159 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:06.611543 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a005530a-997d-45af-9103-3076e22a9159 None admin] REQUEST_METHOD: `PUT` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:06.611924 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a005530a-997d-45af-9103-3076e22a9159 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:06.612302 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a005530a-997d-45af-9103-3076e22a9159 None admin] PATH_INFO: `/v3/projects/33cf2502c8694374bd447d6304e658c2/users/6832ba6fe7d34e438a50d7f4d5681d00/roles/4c9fc0b3baed441da2126be41cc6b04a` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:06.615526 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a005530a-997d-45af-9103-3076e22a9159 None admin] RBAC: Authorizing `identity:create_grant(project_id=33cf2502c8694374bd447d6304e658c2, user_id=6832ba6fe7d34e438a50d7f4d5681d00, role_id=4c9fc0b3baed441da2126be41cc6b04a)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:06.617438 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a005530a-997d-45af-9103-3076e22a9159 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:06.620641 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765dcf98-247b-437e-a3ab-1eb64bddd3b5 None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:06.636180 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 64/126] 10.222.0.36 () {58 vars in 1439 bytes} [Wed Jul 27 10:34:06 2022] PUT /identity/v3/projects/33cf2502c8694374bd447d6304e658c2/users/6832ba6fe7d34e438a50d7f4d5681d00/roles/4c9fc0b3baed441da2126be41cc6b04a => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:34:06.637298 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a005530a-997d-45af-9103-3076e22a9159 None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:06.671052 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebee1e50-f79a-40e6-a57c-5f86bbfc9b83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:06.672071 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebee1e50-f79a-40e6-a57c-5f86bbfc9b83 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:06.672640 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebee1e50-f79a-40e6-a57c-5f86bbfc9b83 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:06.673099 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebee1e50-f79a-40e6-a57c-5f86bbfc9b83 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:06.698354 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebee1e50-f79a-40e6-a57c-5f86bbfc9b83 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 10:34:06.713265 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application [None req-ebee1e50-f79a-40e6-a57c-5f86bbfc9b83 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 27 10:34:06.713265 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:34:06.713265 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:34:06.713265 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:34:06.713265 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:34:06.713265 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:34:06.713265 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:34:06.713265 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:34:06.713265 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:34:06.713265 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:34:06.713265 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:34:06.713265 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:34:06.713265 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 10:34:06.713265 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 10:34:06.713265 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 10:34:06.713265 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 10:34:06.714390 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 10:34:06.714390 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 10:34:06.714390 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 10:34:06.714390 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 10:34:06.714390 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:34:06.714390 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:34:06.714390 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:34:06.714390 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:34:06.714390 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:34:06.714390 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:34:06.714390 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:34:06.714390 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application with Lock( Jul 27 10:34:06.714390 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:34:06.714390 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:34:06.714390 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:34:06.714390 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:34:06.715374 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:34:06.715374 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:34:06.715374 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:34:06.715374 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:34:06.715374 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 10:34:06.715374 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 10:34:06.715374 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 10:34:06.715374 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 10:34:06.715374 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 10:34:06.715374 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 10:34:06.715374 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 27 10:34:06.715374 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Jul 27 10:34:06.717545 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 63/127] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jul 27 10:34:06 2022] GET /identity/v3/roles/member => generated 84 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 10:34:06.718629 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebee1e50-f79a-40e6-a57c-5f86bbfc9b83 None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:06.759241 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a005530a-997d-45af-9103-3076e22a9159 None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:06.796318 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:06.797199 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:06.800542 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd7770e-d86f-47c0-9b65-ef27d6a16a44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:06.801181 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd7770e-d86f-47c0-9b65-ef27d6a16a44 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:06.801467 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd7770e-d86f-47c0-9b65-ef27d6a16a44 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:06.801901 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd7770e-d86f-47c0-9b65-ef27d6a16a44 None admin] PATH_INFO: `/v3/roles` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:06.802708 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd7770e-d86f-47c0-9b65-ef27d6a16a44 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86827) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:34:06.803057 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd7770e-d86f-47c0-9b65-ef27d6a16a44 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:06.804682 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd7770e-d86f-47c0-9b65-ef27d6a16a44 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:06.814957 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 65/128] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jul 27 10:34:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:34:06.816026 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd7770e-d86f-47c0-9b65-ef27d6a16a44 None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:06.914516 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebee1e50-f79a-40e6-a57c-5f86bbfc9b83 None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:06.967222 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd7770e-d86f-47c0-9b65-ef27d6a16a44 None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:06.979681 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b70b65-1005-4a22-9434-8f74c3466787 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:06.980338 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b70b65-1005-4a22-9434-8f74c3466787 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:06.980623 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b70b65-1005-4a22-9434-8f74c3466787 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:06.980896 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b70b65-1005-4a22-9434-8f74c3466787 None admin] PATH_INFO: `/v3/projects/33cf2502c8694374bd447d6304e658c2` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:06.982771 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b70b65-1005-4a22-9434-8f74c3466787 None admin] RBAC: Authorizing `identity:get_project(project_id=33cf2502c8694374bd447d6304e658c2)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:06.984247 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b70b65-1005-4a22-9434-8f74c3466787 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:06.986093 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 64/129] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Jul 27 10:34:06 2022] GET /identity/v3/projects/33cf2502c8694374bd447d6304e658c2 => generated 298 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:34:06.989529 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b70b65-1005-4a22-9434-8f74c3466787 None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:07.025663 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cce0330-e479-4173-8020-62566f6450a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:07.026315 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cce0330-e479-4173-8020-62566f6450a4 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:07.026621 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cce0330-e479-4173-8020-62566f6450a4 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:07.026900 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cce0330-e479-4173-8020-62566f6450a4 None admin] PATH_INFO: `/v3/roles` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:07.027639 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cce0330-e479-4173-8020-62566f6450a4 None admin] RBAC: Adding query filter params (name=member) {{(pid=86827) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:34:07.027947 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cce0330-e479-4173-8020-62566f6450a4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:07.029380 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cce0330-e479-4173-8020-62566f6450a4 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:07.044511 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 66/130] 10.222.0.36 () {56 vars in 1101 bytes} [Wed Jul 27 10:34:06 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:34:07.047488 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cce0330-e479-4173-8020-62566f6450a4 None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:07.146515 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:07.175045 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b70b65-1005-4a22-9434-8f74c3466787 None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:07.185129 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:07.224474 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e126a9d4-450d-47e9-ade6-47599f0efeeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:07.225507 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e126a9d4-450d-47e9-ade6-47599f0efeeb None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:07.225805 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e126a9d4-450d-47e9-ade6-47599f0efeeb None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:07.226089 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e126a9d4-450d-47e9-ade6-47599f0efeeb None admin] PATH_INFO: `/v3/users/60b0ea1b90004781b58a0cbfd103ae8f` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:07.229105 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e126a9d4-450d-47e9-ade6-47599f0efeeb None admin] RBAC: Authorizing `identity:get_user(user_id=60b0ea1b90004781b58a0cbfd103ae8f)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:07.230650 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e126a9d4-450d-47e9-ade6-47599f0efeeb None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:07.230754 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cce0330-e479-4173-8020-62566f6450a4 None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:07.232883 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 65/131] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Jul 27 10:34:06 2022] GET /identity/v3/users/60b0ea1b90004781b58a0cbfd103ae8f => generated 247 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:34:07.234562 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e126a9d4-450d-47e9-ade6-47599f0efeeb None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:07.288170 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c77c05b-ea1c-4ae7-9650-dcb3fa2ab47f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:07.288857 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c77c05b-ea1c-4ae7-9650-dcb3fa2ab47f None admin] REQUEST_METHOD: `PUT` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:07.289140 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c77c05b-ea1c-4ae7-9650-dcb3fa2ab47f None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:07.289451 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c77c05b-ea1c-4ae7-9650-dcb3fa2ab47f None admin] PATH_INFO: `/v3/projects/33cf2502c8694374bd447d6304e658c2/users/6832ba6fe7d34e438a50d7f4d5681d00/roles/c3341f95596344309327efff99a20d5c` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:07.293052 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c77c05b-ea1c-4ae7-9650-dcb3fa2ab47f None admin] RBAC: Authorizing `identity:create_grant(project_id=33cf2502c8694374bd447d6304e658c2, user_id=6832ba6fe7d34e438a50d7f4d5681d00, role_id=c3341f95596344309327efff99a20d5c)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:07.294512 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c77c05b-ea1c-4ae7-9650-dcb3fa2ab47f None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:07.319408 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 67/132] 10.222.0.36 () {58 vars in 1439 bytes} [Wed Jul 27 10:34:07 2022] PUT /identity/v3/projects/33cf2502c8694374bd447d6304e658c2/users/6832ba6fe7d34e438a50d7f4d5681d00/roles/c3341f95596344309327efff99a20d5c => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:34:07.320415 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c77c05b-ea1c-4ae7-9650-dcb3fa2ab47f None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:07.408845 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e126a9d4-450d-47e9-ade6-47599f0efeeb None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:07.487300 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4759af1-1eef-4912-a7cb-28e36a3a55bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:07.487888 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4759af1-1eef-4912-a7cb-28e36a3a55bb None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:07.488364 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4759af1-1eef-4912-a7cb-28e36a3a55bb None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:07.488646 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4759af1-1eef-4912-a7cb-28e36a3a55bb None admin] PATH_INFO: `/v3/users/6832ba6fe7d34e438a50d7f4d5681d00` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:07.489924 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4759af1-1eef-4912-a7cb-28e36a3a55bb None admin] RBAC: Authorizing `identity:get_user(user_id=6832ba6fe7d34e438a50d7f4d5681d00)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:07.491443 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4759af1-1eef-4912-a7cb-28e36a3a55bb None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:07.493687 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 66/133] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Jul 27 10:34:07 2022] GET /identity/v3/users/6832ba6fe7d34e438a50d7f4d5681d00 => generated 275 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:34:07.499921 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4759af1-1eef-4912-a7cb-28e36a3a55bb None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:07.541569 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c77c05b-ea1c-4ae7-9650-dcb3fa2ab47f None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:07.574978 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:07.583438 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:07.621453 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2b64aa-1633-45ab-a9f0-1a1106c36571 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:07.623370 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2b64aa-1633-45ab-a9f0-1a1106c36571 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:07.623666 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2b64aa-1633-45ab-a9f0-1a1106c36571 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:07.623944 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2b64aa-1633-45ab-a9f0-1a1106c36571 None admin] PATH_INFO: `/v3/projects/33cf2502c8694374bd447d6304e658c2` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:07.625265 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2b64aa-1633-45ab-a9f0-1a1106c36571 None admin] RBAC: Authorizing `identity:get_project(project_id=33cf2502c8694374bd447d6304e658c2)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:07.626766 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2b64aa-1633-45ab-a9f0-1a1106c36571 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:07.628695 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 68/134] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Jul 27 10:34:07 2022] GET /identity/v3/projects/33cf2502c8694374bd447d6304e658c2 => generated 298 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:34:07.637318 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2b64aa-1633-45ab-a9f0-1a1106c36571 None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:07.675366 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4759af1-1eef-4912-a7cb-28e36a3a55bb None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:07.726548 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1d7631-26e7-43dd-b829-a68670a1adb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:07.728897 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1d7631-26e7-43dd-b829-a68670a1adb7 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:07.729198 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1d7631-26e7-43dd-b829-a68670a1adb7 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:07.729481 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1d7631-26e7-43dd-b829-a68670a1adb7 None admin] PATH_INFO: `/v3/projects/cb7449fbc55341b2a1a7bb436dcd7b35` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:07.730749 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1d7631-26e7-43dd-b829-a68670a1adb7 None admin] RBAC: Authorizing `identity:get_project(project_id=cb7449fbc55341b2a1a7bb436dcd7b35)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:07.732231 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1d7631-26e7-43dd-b829-a68670a1adb7 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:07.737979 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 67/135] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Jul 27 10:34:07 2022] GET /identity/v3/projects/cb7449fbc55341b2a1a7bb436dcd7b35 => generated 312 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:34:07.737979 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847bff11-1cc0-4518-bec5-04f00432e915 None None] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:07.737979 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847bff11-1cc0-4518-bec5-04f00432e915 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:07.737979 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847bff11-1cc0-4518-bec5-04f00432e915 None None] PATH_INFO: `/` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:07.737979 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 68/136] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 10:34:07 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 10:34:07.740842 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847bff11-1cc0-4518-bec5-04f00432e915 None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:07.766513 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2b64aa-1633-45ab-a9f0-1a1106c36571 None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:07.828748 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542b6248-8764-4cdf-bb18-e9abbd7cbdb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:07.829530 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542b6248-8764-4cdf-bb18-e9abbd7cbdb6 None admin] REQUEST_METHOD: `PUT` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:07.829820 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542b6248-8764-4cdf-bb18-e9abbd7cbdb6 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:07.830098 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542b6248-8764-4cdf-bb18-e9abbd7cbdb6 None admin] PATH_INFO: `/v3/projects/33cf2502c8694374bd447d6304e658c2/users/60b0ea1b90004781b58a0cbfd103ae8f/roles/21156560eb6b436bb567a40b4976f3f4` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:07.835398 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542b6248-8764-4cdf-bb18-e9abbd7cbdb6 None admin] RBAC: Authorizing `identity:create_grant(project_id=33cf2502c8694374bd447d6304e658c2, user_id=60b0ea1b90004781b58a0cbfd103ae8f, role_id=21156560eb6b436bb567a40b4976f3f4)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:07.838298 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542b6248-8764-4cdf-bb18-e9abbd7cbdb6 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:07.862154 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 69/137] 10.222.0.36 () {58 vars in 1439 bytes} [Wed Jul 27 10:34:07 2022] PUT /identity/v3/projects/33cf2502c8694374bd447d6304e658c2/users/60b0ea1b90004781b58a0cbfd103ae8f/roles/21156560eb6b436bb567a40b4976f3f4 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:34:07.864088 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9324d6-5297-46de-91f5-31ce02a9f285 None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:07.864382 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9324d6-5297-46de-91f5-31ce02a9f285 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:07.864664 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9324d6-5297-46de-91f5-31ce02a9f285 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:07.889144 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-7f9324d6-5297-46de-91f5-31ce02a9f285 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:34:07.891159 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-7f9324d6-5297-46de-91f5-31ce02a9f285 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:34:07.930893 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847bff11-1cc0-4518-bec5-04f00432e915 None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:07.936911 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:07.968611 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:07.994891 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 70/138] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 10:34:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 10:34:08.018128 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9324d6-5297-46de-91f5-31ce02a9f285 None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:08.030099 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49855f90-2b56-44d1-8ded-bb82ce7b7233 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:08.030827 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49855f90-2b56-44d1-8ded-bb82ce7b7233 None admin] REQUEST_METHOD: `PUT` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:08.031445 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49855f90-2b56-44d1-8ded-bb82ce7b7233 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:08.031734 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49855f90-2b56-44d1-8ded-bb82ce7b7233 None admin] PATH_INFO: `/v3/projects/cb7449fbc55341b2a1a7bb436dcd7b35/users/6832ba6fe7d34e438a50d7f4d5681d00/roles/4c9fc0b3baed441da2126be41cc6b04a` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:08.036202 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49855f90-2b56-44d1-8ded-bb82ce7b7233 None admin] RBAC: Authorizing `identity:create_grant(project_id=cb7449fbc55341b2a1a7bb436dcd7b35, user_id=6832ba6fe7d34e438a50d7f4d5681d00, role_id=4c9fc0b3baed441da2126be41cc6b04a)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:08.037778 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49855f90-2b56-44d1-8ded-bb82ce7b7233 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:08.063157 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 69/139] 10.222.0.36 () {58 vars in 1439 bytes} [Wed Jul 27 10:34:07 2022] PUT /identity/v3/projects/cb7449fbc55341b2a1a7bb436dcd7b35/users/6832ba6fe7d34e438a50d7f4d5681d00/roles/4c9fc0b3baed441da2126be41cc6b04a => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 10:34:08.168307 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9324d6-5297-46de-91f5-31ce02a9f285 None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:08.209152 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbf322c-cd77-4537-8b49-e2a4c24183ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:08.209828 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbf322c-cd77-4537-8b49-e2a4c24183ea None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:08.210172 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbf322c-cd77-4537-8b49-e2a4c24183ea None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:08.210456 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbf322c-cd77-4537-8b49-e2a4c24183ea None admin] PATH_INFO: `/v3/domains/default` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:08.212288 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cbf322c-cd77-4537-8b49-e2a4c24183ea None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:08.213820 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cbf322c-cd77-4537-8b49-e2a4c24183ea None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:08.216834 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 71/140] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 10:34:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:34:08.219072 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4ea206-61b9-40b5-9fcb-c19ef129284f None None] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:08.219362 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4ea206-61b9-40b5-9fcb-c19ef129284f None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:08.219705 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4ea206-61b9-40b5-9fcb-c19ef129284f None None] PATH_INFO: `/` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:08.221147 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 70/141] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 10:34:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 10:34:08.223601 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbf322c-cd77-4537-8b49-e2a4c24183ea None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:08.228374 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68a7c48-b8af-443d-bef4-3cac3d9a6a6a None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:08.228794 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68a7c48-b8af-443d-bef4-3cac3d9a6a6a None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:08.229171 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68a7c48-b8af-443d-bef4-3cac3d9a6a6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:08.263808 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-f68a7c48-b8af-443d-bef4-3cac3d9a6a6a None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:34:08.265255 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-f68a7c48-b8af-443d-bef4-3cac3d9a6a6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:34:08.293468 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:08.377427 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 71/142] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 10:34:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 27 10:34:08.385660 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbf322c-cd77-4537-8b49-e2a4c24183ea None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:08.417468 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68a7c48-b8af-443d-bef4-3cac3d9a6a6a None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:08.424255 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc62088-6e81-46a8-8e78-b517dc34b689 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:08.425143 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc62088-6e81-46a8-8e78-b517dc34b689 None admin] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:08.425629 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc62088-6e81-46a8-8e78-b517dc34b689 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:08.426128 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc62088-6e81-46a8-8e78-b517dc34b689 None admin] PATH_INFO: `/v3/users` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:08.427055 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc62088-6e81-46a8-8e78-b517dc34b689 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:08.428728 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc62088-6e81-46a8-8e78-b517dc34b689 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:08.460326 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 72/143] 10.222.0.36 () {60 vars in 1132 bytes} [Wed Jul 27 10:34:08 2022] POST /identity/v3/users => generated 283 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 10:34:08.592102 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68a7c48-b8af-443d-bef4-3cac3d9a6a6a None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:08.593686 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:08.647902 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a1a609-1926-4e4e-9feb-c4f149eea17b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:08.648579 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a1a609-1926-4e4e-9feb-c4f149eea17b None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:08.649409 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a1a609-1926-4e4e-9feb-c4f149eea17b None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:08.649687 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a1a609-1926-4e4e-9feb-c4f149eea17b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:08.675825 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a1a609-1926-4e4e-9feb-c4f149eea17b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 10:34:08.681866 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application [None req-71a1a609-1926-4e4e-9feb-c4f149eea17b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 27 10:34:08.681866 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:34:08.681866 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:34:08.681866 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:34:08.681866 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:34:08.681866 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:34:08.681866 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:34:08.681866 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:34:08.681866 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:34:08.681866 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:34:08.681866 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:34:08.681866 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:34:08.681866 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 10:34:08.681866 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 10:34:08.681866 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 10:34:08.681866 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 10:34:08.682759 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 10:34:08.682759 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 10:34:08.682759 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 10:34:08.682759 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 10:34:08.682759 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:34:08.682759 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:34:08.682759 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:34:08.682759 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:34:08.682759 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:34:08.682759 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:34:08.682759 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:34:08.682759 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application with Lock( Jul 27 10:34:08.682759 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:34:08.682759 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:34:08.682759 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:34:08.682759 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:34:08.683525 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:34:08.683525 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:34:08.683525 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:34:08.683525 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:34:08.683525 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 10:34:08.683525 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 10:34:08.683525 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 10:34:08.683525 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 10:34:08.683525 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 10:34:08.683525 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 10:34:08.683525 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 27 10:34:08.683525 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Jul 27 10:34:08.683525 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 72/144] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jul 27 10:34:08 2022] GET /identity/v3/roles/reader => generated 84 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 10:34:08.690991 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc62088-6e81-46a8-8e78-b517dc34b689 None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:08.877988 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:08.886405 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc62088-6e81-46a8-8e78-b517dc34b689 None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:08.930880 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95690f61-d316-494b-a2a5-52db0a82d3d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:08.931590 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95690f61-d316-494b-a2a5-52db0a82d3d0 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:08.931881 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95690f61-d316-494b-a2a5-52db0a82d3d0 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:08.932160 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95690f61-d316-494b-a2a5-52db0a82d3d0 None admin] PATH_INFO: `/v3/roles` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:08.932905 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95690f61-d316-494b-a2a5-52db0a82d3d0 None admin] RBAC: Adding query filter params (name=reader) {{(pid=86827) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:34:08.933216 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95690f61-d316-494b-a2a5-52db0a82d3d0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:08.934628 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95690f61-d316-494b-a2a5-52db0a82d3d0 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:08.944220 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 73/145] 10.222.0.36 () {56 vars in 1101 bytes} [Wed Jul 27 10:34:08 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:34:08.955868 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a1a609-1926-4e4e-9feb-c4f149eea17b None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:09.107971 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a1a609-1926-4e4e-9feb-c4f149eea17b None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:09.156896 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98826eec-27c5-4a39-a987-f6f92ef455f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:09.157468 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98826eec-27c5-4a39-a987-f6f92ef455f1 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:09.157747 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98826eec-27c5-4a39-a987-f6f92ef455f1 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:09.158021 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98826eec-27c5-4a39-a987-f6f92ef455f1 None admin] PATH_INFO: `/v3/users/aa21fc3e363e487d94f11e274b824cf8` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:09.159584 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98826eec-27c5-4a39-a987-f6f92ef455f1 None admin] RBAC: Authorizing `identity:get_user(user_id=aa21fc3e363e487d94f11e274b824cf8)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:09.161055 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98826eec-27c5-4a39-a987-f6f92ef455f1 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:09.163922 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 73/146] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Jul 27 10:34:08 2022] GET /identity/v3/users/aa21fc3e363e487d94f11e274b824cf8 => generated 289 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:34:09.169067 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95690f61-d316-494b-a2a5-52db0a82d3d0 None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:09.306544 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95690f61-d316-494b-a2a5-52db0a82d3d0 None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:09.346174 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130d62e8-6f65-4d27-8d1d-57d8674cb6e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:09.346867 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130d62e8-6f65-4d27-8d1d-57d8674cb6e0 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:09.347156 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130d62e8-6f65-4d27-8d1d-57d8674cb6e0 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:09.347437 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130d62e8-6f65-4d27-8d1d-57d8674cb6e0 None admin] PATH_INFO: `/v3/projects/33cf2502c8694374bd447d6304e658c2` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:09.348772 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130d62e8-6f65-4d27-8d1d-57d8674cb6e0 None admin] RBAC: Authorizing `identity:get_project(project_id=33cf2502c8694374bd447d6304e658c2)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:09.350262 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130d62e8-6f65-4d27-8d1d-57d8674cb6e0 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:09.352870 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 74/147] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Jul 27 10:34:09 2022] GET /identity/v3/projects/33cf2502c8694374bd447d6304e658c2 => generated 298 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:34:09.358526 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98826eec-27c5-4a39-a987-f6f92ef455f1 None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:09.407246 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:09.452867 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14dff183-4f5b-4bee-808d-6638e47393f3 None None] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:09.452867 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14dff183-4f5b-4bee-808d-6638e47393f3 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:09.453214 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14dff183-4f5b-4bee-808d-6638e47393f3 None None] PATH_INFO: `/` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:09.454531 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 75/148] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 10:34:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 10:34:09.459780 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7d9f59-42c9-423e-9499-680b6bc3a9bf None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:09.460070 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7d9f59-42c9-423e-9499-680b6bc3a9bf None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:09.460357 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7d9f59-42c9-423e-9499-680b6bc3a9bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:09.486852 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-af7d9f59-42c9-423e-9499-680b6bc3a9bf None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:34:09.488449 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-af7d9f59-42c9-423e-9499-680b6bc3a9bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:34:09.489712 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98826eec-27c5-4a39-a987-f6f92ef455f1 None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:09.515986 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:09.534185 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf3c9ee-28d3-46bf-9e53-50bcd37cbdde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:09.535041 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf3c9ee-28d3-46bf-9e53-50bcd37cbdde None admin] REQUEST_METHOD: `PUT` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:09.535430 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf3c9ee-28d3-46bf-9e53-50bcd37cbdde None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:09.536314 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf3c9ee-28d3-46bf-9e53-50bcd37cbdde None admin] PATH_INFO: `/v3/projects/33cf2502c8694374bd447d6304e658c2/users/aa21fc3e363e487d94f11e274b824cf8/roles/80f67bfde4f5494e9c6d825aafb44e11` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:09.538038 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf3c9ee-28d3-46bf-9e53-50bcd37cbdde None admin] RBAC: Authorizing `identity:create_grant(project_id=33cf2502c8694374bd447d6304e658c2, user_id=aa21fc3e363e487d94f11e274b824cf8, role_id=80f67bfde4f5494e9c6d825aafb44e11)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:09.540184 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf3c9ee-28d3-46bf-9e53-50bcd37cbdde None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:09.556941 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 74/149] 10.222.0.36 () {58 vars in 1439 bytes} [Wed Jul 27 10:34:09 2022] PUT /identity/v3/projects/33cf2502c8694374bd447d6304e658c2/users/aa21fc3e363e487d94f11e274b824cf8/roles/80f67bfde4f5494e9c6d825aafb44e11 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:34:09.581103 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 76/150] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 10:34:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 27 10:34:09.610136 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf3c9ee-28d3-46bf-9e53-50bcd37cbdde None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:09.731122 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:09.745301 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf3c9ee-28d3-46bf-9e53-50bcd37cbdde None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:09.793858 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5796f6-4b87-42e0-8c44-a635d2090ab4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:09.794556 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5796f6-4b87-42e0-8c44-a635d2090ab4 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:09.794872 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5796f6-4b87-42e0-8c44-a635d2090ab4 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:09.795154 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5796f6-4b87-42e0-8c44-a635d2090ab4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:09.808398 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5796f6-4b87-42e0-8c44-a635d2090ab4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 10:34:09.817817 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application [None req-8e5796f6-4b87-42e0-8c44-a635d2090ab4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 27 10:34:09.817817 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:34:09.817817 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:34:09.817817 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:34:09.817817 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:34:09.817817 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:34:09.817817 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:34:09.817817 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:34:09.817817 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:34:09.817817 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:34:09.817817 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:34:09.817817 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:34:09.817817 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 10:34:09.817817 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 10:34:09.817817 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 10:34:09.817817 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 10:34:09.818887 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 10:34:09.818887 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 10:34:09.818887 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 10:34:09.818887 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 10:34:09.818887 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:34:09.818887 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:34:09.818887 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:34:09.818887 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:34:09.818887 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:34:09.818887 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:34:09.818887 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:34:09.818887 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application with Lock( Jul 27 10:34:09.818887 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:34:09.818887 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:34:09.818887 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:34:09.818887 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:34:09.819683 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:34:09.819683 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:34:09.819683 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:34:09.819683 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:34:09.819683 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 10:34:09.819683 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 10:34:09.819683 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 10:34:09.819683 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 10:34:09.819683 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 10:34:09.819683 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 10:34:09.819683 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 27 10:34:09.819683 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Jul 27 10:34:09.819683 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 75/151] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jul 27 10:34:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 10:34:09.827088 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7d9f59-42c9-423e-9499-680b6bc3a9bf None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:09.960442 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7d9f59-42c9-423e-9499-680b6bc3a9bf None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:10.010184 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68f5ae5-be51-4f9d-84bd-823b452f4022 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:10.010764 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68f5ae5-be51-4f9d-84bd-823b452f4022 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:10.011053 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68f5ae5-be51-4f9d-84bd-823b452f4022 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:10.011332 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68f5ae5-be51-4f9d-84bd-823b452f4022 None admin] PATH_INFO: `/v3/roles` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:10.012066 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68f5ae5-be51-4f9d-84bd-823b452f4022 None admin] RBAC: Adding query filter params (name=member) {{(pid=86827) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:34:10.012379 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68f5ae5-be51-4f9d-84bd-823b452f4022 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:10.013776 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68f5ae5-be51-4f9d-84bd-823b452f4022 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:10.022569 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 77/152] 10.222.0.36 () {56 vars in 1101 bytes} [Wed Jul 27 10:34:09 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:34:10.028585 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5796f6-4b87-42e0-8c44-a635d2090ab4 None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:10.141850 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5796f6-4b87-42e0-8c44-a635d2090ab4 None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:10.184524 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7152ca0-e918-4619-8d63-dfd3d500c371 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:10.185364 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7152ca0-e918-4619-8d63-dfd3d500c371 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:10.185657 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7152ca0-e918-4619-8d63-dfd3d500c371 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:10.185936 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7152ca0-e918-4619-8d63-dfd3d500c371 None admin] PATH_INFO: `/v3/users/6832ba6fe7d34e438a50d7f4d5681d00` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:10.187185 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7152ca0-e918-4619-8d63-dfd3d500c371 None admin] RBAC: Authorizing `identity:get_user(user_id=6832ba6fe7d34e438a50d7f4d5681d00)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:10.188727 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7152ca0-e918-4619-8d63-dfd3d500c371 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:10.190437 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 76/153] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Jul 27 10:34:10 2022] GET /identity/v3/users/6832ba6fe7d34e438a50d7f4d5681d00 => generated 275 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:34:10.196085 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68f5ae5-be51-4f9d-84bd-823b452f4022 None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:10.217399 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:10.308085 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68f5ae5-be51-4f9d-84bd-823b452f4022 None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:10.344053 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb6ca90-3304-4cd6-aedb-d3ad0db7e695 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:10.344693 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb6ca90-3304-4cd6-aedb-d3ad0db7e695 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:10.344973 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb6ca90-3304-4cd6-aedb-d3ad0db7e695 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:10.345256 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb6ca90-3304-4cd6-aedb-d3ad0db7e695 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:10.419038 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb6ca90-3304-4cd6-aedb-d3ad0db7e695 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:10.420526 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb6ca90-3304-4cd6-aedb-d3ad0db7e695 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:10.443001 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:10.525838 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 78/154] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jul 27 10:34:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 10:34:10.531427 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7152ca0-e918-4619-8d63-dfd3d500c371 None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:10.552469 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:10.639505 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7152ca0-e918-4619-8d63-dfd3d500c371 None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:10.683344 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d36269-fa74-47a0-bf08-3385c19e1ff5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:10.684020 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d36269-fa74-47a0-bf08-3385c19e1ff5 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:10.684304 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d36269-fa74-47a0-bf08-3385c19e1ff5 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:10.684583 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d36269-fa74-47a0-bf08-3385c19e1ff5 None admin] PATH_INFO: `/v3/projects/33cf2502c8694374bd447d6304e658c2` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:10.686022 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d36269-fa74-47a0-bf08-3385c19e1ff5 None admin] RBAC: Authorizing `identity:get_project(project_id=33cf2502c8694374bd447d6304e658c2)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:10.687542 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d36269-fa74-47a0-bf08-3385c19e1ff5 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:10.689729 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 77/155] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Jul 27 10:34:10 2022] GET /identity/v3/projects/33cf2502c8694374bd447d6304e658c2 => generated 298 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:34:10.695735 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb6ca90-3304-4cd6-aedb-d3ad0db7e695 None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:10.804721 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb6ca90-3304-4cd6-aedb-d3ad0db7e695 None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:10.831845 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:10.841987 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f76c04e-2813-4c1e-ae9b-0e5203547f7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:10.842646 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f76c04e-2813-4c1e-ae9b-0e5203547f7b None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:10.842938 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f76c04e-2813-4c1e-ae9b-0e5203547f7b None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:10.843218 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f76c04e-2813-4c1e-ae9b-0e5203547f7b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:10.844034 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f76c04e-2813-4c1e-ae9b-0e5203547f7b None admin] RBAC: Adding query filter params (role.id=4c9fc0b3baed441da2126be41cc6b04a, scope.project.id=33cf2502c8694374bd447d6304e658c2, user.id=6832ba6fe7d34e438a50d7f4d5681d00) {{(pid=86827) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:34:10.844350 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f76c04e-2813-4c1e-ae9b-0e5203547f7b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:10.845817 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f76c04e-2813-4c1e-ae9b-0e5203547f7b None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:10.853108 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 79/156] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Jul 27 10:34:10 2022] GET /identity/v3/role_assignments?user.id=6832ba6fe7d34e438a50d7f4d5681d00&scope.project.id=33cf2502c8694374bd447d6304e658c2&role.id=4c9fc0b3baed441da2126be41cc6b04a => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:34:10.984156 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ff0cbe-c9cd-40cd-8de8-7c99bb1fb101 None None] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:10.984488 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ff0cbe-c9cd-40cd-8de8-7c99bb1fb101 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:10.984723 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ff0cbe-c9cd-40cd-8de8-7c99bb1fb101 None None] PATH_INFO: `/` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:10.986120 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 78/157] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 10:34:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 10:34:11.004260 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bb74b1-72e0-42b3-9ed0-7c6916824c7e None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:11.004643 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bb74b1-72e0-42b3-9ed0-7c6916824c7e None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:11.004825 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bb74b1-72e0-42b3-9ed0-7c6916824c7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:11.027075 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-31bb74b1-72e0-42b3-9ed0-7c6916824c7e None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:34:11.028322 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-31bb74b1-72e0-42b3-9ed0-7c6916824c7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:34:11.103750 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 80/158] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 10:34:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 10:34:11.151322 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ff0cbe-c9cd-40cd-8de8-7c99bb1fb101 None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:11.249108 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:11.267146 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ff0cbe-c9cd-40cd-8de8-7c99bb1fb101 None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:11.309124 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34f717c-e309-416e-8138-2172d5c7fc41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:11.310161 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34f717c-e309-416e-8138-2172d5c7fc41 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:11.310697 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34f717c-e309-416e-8138-2172d5c7fc41 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:11.311220 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34f717c-e309-416e-8138-2172d5c7fc41 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:11.323305 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34f717c-e309-416e-8138-2172d5c7fc41 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 10:34:11.328615 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application [None req-d34f717c-e309-416e-8138-2172d5c7fc41 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 27 10:34:11.328615 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:34:11.328615 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:34:11.328615 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:34:11.328615 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:34:11.328615 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:34:11.328615 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:34:11.328615 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:34:11.328615 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:34:11.328615 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:34:11.328615 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:34:11.328615 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:34:11.328615 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 10:34:11.328615 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 10:34:11.328615 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 10:34:11.328615 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 10:34:11.329522 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 10:34:11.329522 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 10:34:11.329522 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 10:34:11.329522 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 10:34:11.329522 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:34:11.329522 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:34:11.329522 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:34:11.329522 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:34:11.329522 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:34:11.329522 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:34:11.329522 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:34:11.329522 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application with Lock( Jul 27 10:34:11.329522 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:34:11.329522 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:34:11.329522 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:34:11.329522 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:34:11.330287 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:34:11.330287 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:34:11.330287 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:34:11.330287 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:34:11.330287 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 10:34:11.330287 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 10:34:11.330287 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 10:34:11.330287 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 10:34:11.330287 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 10:34:11.330287 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 10:34:11.330287 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 27 10:34:11.330287 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Jul 27 10:34:11.332598 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 79/159] 10.222.0.36 () {56 vars in 1114 bytes} [Wed Jul 27 10:34:11 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 10:34:11.339256 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bb74b1-72e0-42b3-9ed0-7c6916824c7e None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:11.471489 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bb74b1-72e0-42b3-9ed0-7c6916824c7e None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:11.487373 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:11.510622 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7b409f-d07c-4c0a-a8d3-e616a51d77e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:11.511249 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7b409f-d07c-4c0a-a8d3-e616a51d77e5 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:11.511546 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7b409f-d07c-4c0a-a8d3-e616a51d77e5 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:11.511825 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7b409f-d07c-4c0a-a8d3-e616a51d77e5 None admin] PATH_INFO: `/v3/roles` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:11.512571 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7b409f-d07c-4c0a-a8d3-e616a51d77e5 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86827) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:34:11.512935 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7b409f-d07c-4c0a-a8d3-e616a51d77e5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:11.514399 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7b409f-d07c-4c0a-a8d3-e616a51d77e5 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:11.524286 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 81/160] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Jul 27 10:34:11 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:34:11.529729 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34f717c-e309-416e-8138-2172d5c7fc41 None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:11.685718 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34f717c-e309-416e-8138-2172d5c7fc41 None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:11.741692 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b79a17-12ca-407c-8678-2b58e3d32044 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:11.742367 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b79a17-12ca-407c-8678-2b58e3d32044 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:11.742673 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b79a17-12ca-407c-8678-2b58e3d32044 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:11.742949 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b79a17-12ca-407c-8678-2b58e3d32044 None admin] PATH_INFO: `/v3/users/6832ba6fe7d34e438a50d7f4d5681d00` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:11.747319 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b79a17-12ca-407c-8678-2b58e3d32044 None admin] RBAC: Authorizing `identity:get_user(user_id=6832ba6fe7d34e438a50d7f4d5681d00)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:11.748799 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b79a17-12ca-407c-8678-2b58e3d32044 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:11.750433 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 80/161] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Jul 27 10:34:11 2022] GET /identity/v3/users/6832ba6fe7d34e438a50d7f4d5681d00 => generated 275 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:34:11.760039 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7b409f-d07c-4c0a-a8d3-e616a51d77e5 None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:11.792988 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e0c5b7-b6fe-42c5-b0d0-f2e23a9375cd None None] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:11.793310 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e0c5b7-b6fe-42c5-b0d0-f2e23a9375cd None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:11.793484 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e0c5b7-b6fe-42c5-b0d0-f2e23a9375cd None None] PATH_INFO: `/` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:11.794837 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 81/162] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 10:34:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 10:34:11.799860 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efe1def-8021-4c2d-aaae-569e85c58595 None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:11.800145 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efe1def-8021-4c2d-aaae-569e85c58595 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:11.800421 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efe1def-8021-4c2d-aaae-569e85c58595 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:11.843600 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-9efe1def-8021-4c2d-aaae-569e85c58595 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:34:11.844806 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-9efe1def-8021-4c2d-aaae-569e85c58595 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:34:11.860668 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:11.884239 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7b409f-d07c-4c0a-a8d3-e616a51d77e5 None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:11.920483 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a7375e-39ee-4107-b389-054815c9c197 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:11.921120 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a7375e-39ee-4107-b389-054815c9c197 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:11.921413 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a7375e-39ee-4107-b389-054815c9c197 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:11.921696 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a7375e-39ee-4107-b389-054815c9c197 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:11.967005 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:11.980553 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 82/163] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 10:34:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 10:34:11.981870 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4e5062-be77-4bf8-aa31-3abf1dac37b0 None None] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:11.982139 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4e5062-be77-4bf8-aa31-3abf1dac37b0 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:11.982418 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4e5062-be77-4bf8-aa31-3abf1dac37b0 None None] PATH_INFO: `/` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:11.983699 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 83/164] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 10:34:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 10:34:11.991842 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9822096-16aa-4b5b-bfdf-75a7ced60489 None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:11.992127 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9822096-16aa-4b5b-bfdf-75a7ced60489 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:11.992458 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9822096-16aa-4b5b-bfdf-75a7ced60489 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:11.994476 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a7375e-39ee-4107-b389-054815c9c197 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:11.996377 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a7375e-39ee-4107-b389-054815c9c197 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:12.018051 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-a9822096-16aa-4b5b-bfdf-75a7ced60489 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:34:12.031478 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-a9822096-16aa-4b5b-bfdf-75a7ced60489 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:34:12.103587 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 82/165] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jul 27 10:34:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 10:34:12.104744 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a7375e-39ee-4107-b389-054815c9c197 None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:12.144946 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 84/166] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 10:34:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 10:34:12.145823 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9822096-16aa-4b5b-bfdf-75a7ced60489 None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:12.190987 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:12.214290 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a7375e-39ee-4107-b389-054815c9c197 None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:12.261113 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6fb81c-9056-4524-91b0-78c3e95884b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:12.262036 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6fb81c-9056-4524-91b0-78c3e95884b2 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:12.262474 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6fb81c-9056-4524-91b0-78c3e95884b2 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:12.262915 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6fb81c-9056-4524-91b0-78c3e95884b2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:12.274168 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6fb81c-9056-4524-91b0-78c3e95884b2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 10:34:12.281357 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application [None req-9f6fb81c-9056-4524-91b0-78c3e95884b2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 27 10:34:12.281357 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:34:12.281357 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:34:12.281357 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:34:12.281357 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:34:12.281357 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:34:12.281357 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:34:12.281357 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:34:12.281357 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:34:12.281357 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:34:12.281357 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:34:12.281357 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:34:12.281357 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 10:34:12.281357 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 10:34:12.281357 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 10:34:12.281357 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 10:34:12.282418 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 10:34:12.282418 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 10:34:12.282418 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 10:34:12.282418 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 10:34:12.282418 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:34:12.282418 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:34:12.282418 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:34:12.282418 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:34:12.282418 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:34:12.282418 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:34:12.282418 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:34:12.282418 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application with Lock( Jul 27 10:34:12.282418 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:34:12.282418 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:34:12.282418 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:34:12.282418 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:34:12.283570 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:34:12.283570 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:34:12.283570 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:34:12.283570 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:34:12.283570 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 10:34:12.283570 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 10:34:12.283570 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 10:34:12.283570 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 10:34:12.283570 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 10:34:12.283570 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 10:34:12.283570 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 27 10:34:12.283570 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Jul 27 10:34:12.285690 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 83/167] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jul 27 10:34:12 2022] GET /identity/v3/roles/member => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 10:34:12.286636 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6fb81c-9056-4524-91b0-78c3e95884b2 None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:12.300775 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:12.351314 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9822096-16aa-4b5b-bfdf-75a7ced60489 None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:12.400774 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6fb81c-9056-4524-91b0-78c3e95884b2 None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:12.423803 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3406eb-959f-407c-b3f6-fb25caea90d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:12.424474 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3406eb-959f-407c-b3f6-fb25caea90d2 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:12.424769 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3406eb-959f-407c-b3f6-fb25caea90d2 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:12.425096 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3406eb-959f-407c-b3f6-fb25caea90d2 None admin] PATH_INFO: `/v3/projects/33cf2502c8694374bd447d6304e658c2` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:12.426293 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3406eb-959f-407c-b3f6-fb25caea90d2 None admin] RBAC: Authorizing `identity:get_project(project_id=33cf2502c8694374bd447d6304e658c2)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:12.428009 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3406eb-959f-407c-b3f6-fb25caea90d2 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:12.429714 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 85/168] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Jul 27 10:34:12 2022] GET /identity/v3/projects/33cf2502c8694374bd447d6304e658c2 => generated 298 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:34:12.431132 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3406eb-959f-407c-b3f6-fb25caea90d2 None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:12.439659 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b682d73-c6d9-448c-ba59-2b090074ffb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:12.440538 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b682d73-c6d9-448c-ba59-2b090074ffb8 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:12.441027 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b682d73-c6d9-448c-ba59-2b090074ffb8 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:12.441552 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b682d73-c6d9-448c-ba59-2b090074ffb8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:12.443718 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:12.454450 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b682d73-c6d9-448c-ba59-2b090074ffb8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 10:34:12.460166 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application [None req-4b682d73-c6d9-448c-ba59-2b090074ffb8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 10:34:12.460166 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:34:12.460166 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:34:12.460166 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:34:12.460166 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:34:12.460166 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:34:12.460166 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:34:12.460166 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:34:12.460166 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:34:12.460166 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:34:12.460166 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:34:12.460166 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:34:12.460166 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 10:34:12.460166 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 10:34:12.460166 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 10:34:12.460166 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 10:34:12.461275 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 10:34:12.461275 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 10:34:12.461275 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 10:34:12.461275 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 10:34:12.461275 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:34:12.461275 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:34:12.461275 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:34:12.461275 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:34:12.461275 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:34:12.461275 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:34:12.461275 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:34:12.461275 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application with Lock( Jul 27 10:34:12.461275 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:34:12.461275 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:34:12.461275 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:34:12.461275 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:34:12.462156 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:34:12.462156 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:34:12.462156 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:34:12.462156 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:34:12.462156 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 10:34:12.462156 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 10:34:12.462156 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 10:34:12.462156 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 10:34:12.462156 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 10:34:12.462156 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 10:34:12.462156 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 10:34:12.462156 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Jul 27 10:34:12.464150 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 84/169] 10.222.0.36 () {56 vars in 1096 bytes} [Wed Jul 27 10:34:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 10:34:12.465057 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b682d73-c6d9-448c-ba59-2b090074ffb8 None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:12.580475 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b682d73-c6d9-448c-ba59-2b090074ffb8 None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:12.619149 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285ccc0f-f5d7-4423-b343-87b30d187a12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:12.620123 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285ccc0f-f5d7-4423-b343-87b30d187a12 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:12.620584 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285ccc0f-f5d7-4423-b343-87b30d187a12 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:12.621083 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285ccc0f-f5d7-4423-b343-87b30d187a12 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:12.622031 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285ccc0f-f5d7-4423-b343-87b30d187a12 None admin] RBAC: Adding query filter params (role.id=c3341f95596344309327efff99a20d5c, scope.project.id=33cf2502c8694374bd447d6304e658c2, user.id=6832ba6fe7d34e438a50d7f4d5681d00) {{(pid=86827) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:34:12.622490 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285ccc0f-f5d7-4423-b343-87b30d187a12 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:12.624107 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285ccc0f-f5d7-4423-b343-87b30d187a12 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:12.630517 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 85/170] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Jul 27 10:34:12 2022] GET /identity/v3/role_assignments?user.id=6832ba6fe7d34e438a50d7f4d5681d00&scope.project.id=33cf2502c8694374bd447d6304e658c2&role.id=c3341f95596344309327efff99a20d5c => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:34:12.631623 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285ccc0f-f5d7-4423-b343-87b30d187a12 None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:12.634337 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3406eb-959f-407c-b3f6-fb25caea90d2 None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:12.703937 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:12.713870 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14408083-4c62-44d8-bb9a-9f4cdc1bae38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:12.715065 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14408083-4c62-44d8-bb9a-9f4cdc1bae38 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:12.715065 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14408083-4c62-44d8-bb9a-9f4cdc1bae38 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:12.715284 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14408083-4c62-44d8-bb9a-9f4cdc1bae38 None admin] PATH_INFO: `/v3/roles` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:12.715932 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14408083-4c62-44d8-bb9a-9f4cdc1bae38 None admin] RBAC: Adding query filter params (name=member) {{(pid=86828) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:34:12.716282 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14408083-4c62-44d8-bb9a-9f4cdc1bae38 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:12.717803 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14408083-4c62-44d8-bb9a-9f4cdc1bae38 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:12.733273 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 86/171] 10.222.0.36 () {56 vars in 1101 bytes} [Wed Jul 27 10:34:12 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:34:12.739191 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14408083-4c62-44d8-bb9a-9f4cdc1bae38 None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:12.747891 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:12.760343 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285ccc0f-f5d7-4423-b343-87b30d187a12 None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:12.802101 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db2b324-11bf-48d5-901b-38ba7d3d9f42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:12.803137 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db2b324-11bf-48d5-901b-38ba7d3d9f42 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:12.804038 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db2b324-11bf-48d5-901b-38ba7d3d9f42 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:12.804492 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db2b324-11bf-48d5-901b-38ba7d3d9f42 None admin] PATH_INFO: `/v3/roles` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:12.806280 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db2b324-11bf-48d5-901b-38ba7d3d9f42 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86827) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:34:12.806803 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db2b324-11bf-48d5-901b-38ba7d3d9f42 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:12.808399 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db2b324-11bf-48d5-901b-38ba7d3d9f42 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:12.819058 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 86/172] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jul 27 10:34:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:34:12.825321 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db2b324-11bf-48d5-901b-38ba7d3d9f42 None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:12.946938 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db2b324-11bf-48d5-901b-38ba7d3d9f42 None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:12.970709 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:12.981273 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14408083-4c62-44d8-bb9a-9f4cdc1bae38 None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:13.000698 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce398aa-0466-4268-acc9-09632933deea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:13.001541 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce398aa-0466-4268-acc9-09632933deea None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:13.001935 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce398aa-0466-4268-acc9-09632933deea None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:13.002322 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce398aa-0466-4268-acc9-09632933deea None admin] PATH_INFO: `/v3/users/60b0ea1b90004781b58a0cbfd103ae8f` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:13.004053 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce398aa-0466-4268-acc9-09632933deea None admin] RBAC: Authorizing `identity:get_user(user_id=60b0ea1b90004781b58a0cbfd103ae8f)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:13.006014 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce398aa-0466-4268-acc9-09632933deea None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:13.008041 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 87/173] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Jul 27 10:34:12 2022] GET /identity/v3/users/60b0ea1b90004781b58a0cbfd103ae8f => generated 247 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 10:34:13.013942 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce398aa-0466-4268-acc9-09632933deea None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:13.098000 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67085994-040b-4c80-8fc8-0b5633b7ac91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:13.099034 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67085994-040b-4c80-8fc8-0b5633b7ac91 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:13.099469 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67085994-040b-4c80-8fc8-0b5633b7ac91 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:13.099897 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67085994-040b-4c80-8fc8-0b5633b7ac91 None admin] PATH_INFO: `/v3/users/6832ba6fe7d34e438a50d7f4d5681d00` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:13.102274 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67085994-040b-4c80-8fc8-0b5633b7ac91 None admin] RBAC: Authorizing `identity:get_user(user_id=6832ba6fe7d34e438a50d7f4d5681d00)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:13.103955 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67085994-040b-4c80-8fc8-0b5633b7ac91 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:13.105962 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 87/174] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Jul 27 10:34:12 2022] GET /identity/v3/users/6832ba6fe7d34e438a50d7f4d5681d00 => generated 275 bytes in 367 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:34:13.111799 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67085994-040b-4c80-8fc8-0b5633b7ac91 None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:13.165608 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce398aa-0466-4268-acc9-09632933deea None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:13.203139 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:13.235800 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919e2e6b-603e-45e7-8fb4-4352520b6b3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:13.236523 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919e2e6b-603e-45e7-8fb4-4352520b6b3c None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:13.236921 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919e2e6b-603e-45e7-8fb4-4352520b6b3c None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:13.237311 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919e2e6b-603e-45e7-8fb4-4352520b6b3c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:13.243261 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67085994-040b-4c80-8fc8-0b5633b7ac91 None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:13.293477 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47217cf2-7eb2-4a29-9e59-aca9a92cd92f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:13.294123 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47217cf2-7eb2-4a29-9e59-aca9a92cd92f None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:13.294410 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47217cf2-7eb2-4a29-9e59-aca9a92cd92f None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:13.294713 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47217cf2-7eb2-4a29-9e59-aca9a92cd92f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:13.313219 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:13.331737 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919e2e6b-603e-45e7-8fb4-4352520b6b3c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:13.333952 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919e2e6b-603e-45e7-8fb4-4352520b6b3c None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:13.375959 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47217cf2-7eb2-4a29-9e59-aca9a92cd92f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:13.377447 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47217cf2-7eb2-4a29-9e59-aca9a92cd92f None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:13.488101 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 88/175] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jul 27 10:34:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 474 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 10:34:13.490104 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd982d9-af21-4c92-a104-da5d0c2f8a7e None None] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:13.493772 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd982d9-af21-4c92-a104-da5d0c2f8a7e None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:13.494218 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd982d9-af21-4c92-a104-da5d0c2f8a7e None None] PATH_INFO: `/` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:13.495766 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 89/176] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 10:34:13 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 10:34:13.501534 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069b90c6-173e-4c3b-ac37-66d323e9fa07 None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:13.502302 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069b90c6-173e-4c3b-ac37-66d323e9fa07 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:13.502757 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069b90c6-173e-4c3b-ac37-66d323e9fa07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:13.515039 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:13.535545 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-069b90c6-173e-4c3b-ac37-66d323e9fa07 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:34:13.537463 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-069b90c6-173e-4c3b-ac37-66d323e9fa07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:34:13.586438 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 88/177] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jul 27 10:34:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 473 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 10:34:13.589151 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47217cf2-7eb2-4a29-9e59-aca9a92cd92f None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:13.658803 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 90/178] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 10:34:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 10:34:13.659628 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069b90c6-173e-4c3b-ac37-66d323e9fa07 None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:13.691625 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:13.712767 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47217cf2-7eb2-4a29-9e59-aca9a92cd92f None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:13.750279 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd484a20-3c0e-454f-8f4f-d05f98a9bc37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:13.750950 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd484a20-3c0e-454f-8f4f-d05f98a9bc37 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:13.751236 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd484a20-3c0e-454f-8f4f-d05f98a9bc37 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:13.751514 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd484a20-3c0e-454f-8f4f-d05f98a9bc37 None admin] PATH_INFO: `/v3/projects/33cf2502c8694374bd447d6304e658c2` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:13.753121 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd484a20-3c0e-454f-8f4f-d05f98a9bc37 None admin] RBAC: Authorizing `identity:get_project(project_id=33cf2502c8694374bd447d6304e658c2)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:13.754986 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd484a20-3c0e-454f-8f4f-d05f98a9bc37 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:13.762122 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 89/179] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Jul 27 10:34:13 2022] GET /identity/v3/projects/33cf2502c8694374bd447d6304e658c2 => generated 298 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:34:13.762122 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04442199-862c-46e3-91e3-faf919c22009 None None] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:13.762122 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04442199-862c-46e3-91e3-faf919c22009 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:13.762122 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04442199-862c-46e3-91e3-faf919c22009 None None] PATH_INFO: `/` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:13.762122 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 90/180] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 10:34:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 10:34:13.762122 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04442199-862c-46e3-91e3-faf919c22009 None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:13.787248 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069b90c6-173e-4c3b-ac37-66d323e9fa07 None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:13.806549 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:13.825998 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9ef723-6c04-4a93-9386-aab45f933b14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:13.826675 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9ef723-6c04-4a93-9386-aab45f933b14 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:13.826963 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9ef723-6c04-4a93-9386-aab45f933b14 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:13.827247 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9ef723-6c04-4a93-9386-aab45f933b14 None admin] PATH_INFO: `/v3/projects/cb7449fbc55341b2a1a7bb436dcd7b35` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:13.828416 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9ef723-6c04-4a93-9386-aab45f933b14 None admin] RBAC: Authorizing `identity:get_project(project_id=cb7449fbc55341b2a1a7bb436dcd7b35)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:13.829877 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9ef723-6c04-4a93-9386-aab45f933b14 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:13.831595 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 91/181] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Jul 27 10:34:13 2022] GET /identity/v3/projects/cb7449fbc55341b2a1a7bb436dcd7b35 => generated 312 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:34:13.831595 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4651dc08-90ed-4797-b732-bc4c4c5f9909 None None] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:13.831595 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4651dc08-90ed-4797-b732-bc4c4c5f9909 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:13.831595 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4651dc08-90ed-4797-b732-bc4c4c5f9909 None None] PATH_INFO: `/` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:13.831595 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 92/182] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 10:34:13 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 10:34:13.831595 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4651dc08-90ed-4797-b732-bc4c4c5f9909 None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:13.877801 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04442199-862c-46e3-91e3-faf919c22009 None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:13.912884 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fcb97a-5b88-4bc5-81c1-5f53b9ea00d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:13.913526 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fcb97a-5b88-4bc5-81c1-5f53b9ea00d5 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:13.913809 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fcb97a-5b88-4bc5-81c1-5f53b9ea00d5 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:13.914081 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fcb97a-5b88-4bc5-81c1-5f53b9ea00d5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:13.925320 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25fcb97a-5b88-4bc5-81c1-5f53b9ea00d5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 10:34:13.927966 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:13.930311 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application [None req-25fcb97a-5b88-4bc5-81c1-5f53b9ea00d5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 10:34:13.930311 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:34:13.930311 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:34:13.930311 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:34:13.930311 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:34:13.930311 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:34:13.930311 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:34:13.930311 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:34:13.930311 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:34:13.930311 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:34:13.930311 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:34:13.930311 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:34:13.930311 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 10:34:13.930311 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 10:34:13.930311 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 10:34:13.930311 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 10:34:13.931330 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 10:34:13.931330 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 10:34:13.931330 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 10:34:13.931330 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 10:34:13.931330 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:34:13.931330 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:34:13.931330 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:34:13.931330 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:34:13.931330 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:34:13.931330 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:34:13.931330 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:34:13.931330 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application with Lock( Jul 27 10:34:13.931330 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:34:13.931330 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:34:13.931330 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:34:13.931330 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:34:13.932133 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:34:13.932133 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:34:13.932133 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:34:13.932133 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:34:13.932133 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 10:34:13.932133 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 10:34:13.932133 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 10:34:13.932133 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 10:34:13.932133 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 10:34:13.932133 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 10:34:13.932133 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 10:34:13.932133 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Jul 27 10:34:13.932133 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 91/183] 10.222.0.36 () {56 vars in 1096 bytes} [Wed Jul 27 10:34:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 10:34:13.932768 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9204a6c6-b964-46a5-9ce8-5739e72190b0 None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:13.932946 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9204a6c6-b964-46a5-9ce8-5739e72190b0 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:13.933194 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9204a6c6-b964-46a5-9ce8-5739e72190b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:13.947210 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4651dc08-90ed-4797-b732-bc4c4c5f9909 None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:13.953774 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-9204a6c6-b964-46a5-9ce8-5739e72190b0 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:34:13.954999 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-9204a6c6-b964-46a5-9ce8-5739e72190b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:34:13.981930 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ce73d3-cd8a-4593-98ba-b968e10fd0e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:13.982546 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ce73d3-cd8a-4593-98ba-b968e10fd0e5 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:13.982856 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ce73d3-cd8a-4593-98ba-b968e10fd0e5 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:13.983133 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ce73d3-cd8a-4593-98ba-b968e10fd0e5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:13.983926 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ce73d3-cd8a-4593-98ba-b968e10fd0e5 None admin] RBAC: Adding query filter params (role.id=21156560eb6b436bb567a40b4976f3f4, scope.project.id=33cf2502c8694374bd447d6304e658c2, user.id=60b0ea1b90004781b58a0cbfd103ae8f) {{(pid=86827) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:34:13.984238 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ce73d3-cd8a-4593-98ba-b968e10fd0e5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:13.985691 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ce73d3-cd8a-4593-98ba-b968e10fd0e5 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:13.991398 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 93/184] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Jul 27 10:34:13 2022] GET /identity/v3/role_assignments?user.id=60b0ea1b90004781b58a0cbfd103ae8f&scope.project.id=33cf2502c8694374bd447d6304e658c2&role.id=21156560eb6b436bb567a40b4976f3f4 => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:34:13.992176 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ce73d3-cd8a-4593-98ba-b968e10fd0e5 None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:14.032677 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 92/185] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 10:34:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 10:34:14.040564 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188e66c0-4e07-4666-9ef9-583bc02bbc53 None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:14.040975 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188e66c0-4e07-4666-9ef9-583bc02bbc53 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:14.041731 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188e66c0-4e07-4666-9ef9-583bc02bbc53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:14.064684 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-188e66c0-4e07-4666-9ef9-583bc02bbc53 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:34:14.065907 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-188e66c0-4e07-4666-9ef9-583bc02bbc53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:34:14.093729 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:14.107176 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ce73d3-cd8a-4593-98ba-b968e10fd0e5 None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:14.146634 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 93/186] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 10:34:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 10:34:14.146634 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff720e39-9802-48b4-b58f-67618fc8bd56 None None] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:14.146634 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff720e39-9802-48b4-b58f-67618fc8bd56 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:14.146634 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff720e39-9802-48b4-b58f-67618fc8bd56 None None] PATH_INFO: `/` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:14.146634 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 94/187] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 10:34:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 10:34:14.146634 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff720e39-9802-48b4-b58f-67618fc8bd56 None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:14.151668 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d8255e-016e-4700-87be-8664e38cb497 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:14.152307 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d8255e-016e-4700-87be-8664e38cb497 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:14.152594 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d8255e-016e-4700-87be-8664e38cb497 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:14.152875 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d8255e-016e-4700-87be-8664e38cb497 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:14.153683 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d8255e-016e-4700-87be-8664e38cb497 None admin] RBAC: Adding query filter params (role.id=4c9fc0b3baed441da2126be41cc6b04a, scope.project.id=cb7449fbc55341b2a1a7bb436dcd7b35, user.id=6832ba6fe7d34e438a50d7f4d5681d00) {{(pid=86827) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:34:14.154001 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d8255e-016e-4700-87be-8664e38cb497 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:14.155489 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d8255e-016e-4700-87be-8664e38cb497 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:14.163434 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 94/188] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Jul 27 10:34:13 2022] GET /identity/v3/role_assignments?user.id=6832ba6fe7d34e438a50d7f4d5681d00&scope.project.id=cb7449fbc55341b2a1a7bb436dcd7b35&role.id=4c9fc0b3baed441da2126be41cc6b04a => generated 608 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:34:14.164395 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d8255e-016e-4700-87be-8664e38cb497 None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:14.182915 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:14.287791 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff720e39-9802-48b4-b58f-67618fc8bd56 None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:14.299412 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d8255e-016e-4700-87be-8664e38cb497 None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:14.329754 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc485028-4832-4922-88e7-585ad4e7fc07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:14.329754 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc485028-4832-4922-88e7-585ad4e7fc07 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:14.330057 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc485028-4832-4922-88e7-585ad4e7fc07 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:14.330227 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc485028-4832-4922-88e7-585ad4e7fc07 None admin] PATH_INFO: `/v3/roles` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:14.330998 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc485028-4832-4922-88e7-585ad4e7fc07 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86828) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:34:14.331308 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc485028-4832-4922-88e7-585ad4e7fc07 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:14.332696 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc485028-4832-4922-88e7-585ad4e7fc07 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:14.342680 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 95/189] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jul 27 10:34:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:34:14.343919 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca7f4c8-b80e-412f-8274-560fd0e97b8c None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:14.344207 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca7f4c8-b80e-412f-8274-560fd0e97b8c None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:14.344484 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca7f4c8-b80e-412f-8274-560fd0e97b8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:14.345736 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49ebd4d-599b-4849-a6c4-66190a5671a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:14.346388 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49ebd4d-599b-4849-a6c4-66190a5671a1 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:14.346719 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49ebd4d-599b-4849-a6c4-66190a5671a1 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:14.347046 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49ebd4d-599b-4849-a6c4-66190a5671a1 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:14.358542 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49ebd4d-599b-4849-a6c4-66190a5671a1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 10:34:14.362937 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-eca7f4c8-b80e-412f-8274-560fd0e97b8c None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:34:14.364089 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application [None req-f49ebd4d-599b-4849-a6c4-66190a5671a1 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 27 10:34:14.364089 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:34:14.364089 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:34:14.364089 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:34:14.364089 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:34:14.364089 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:34:14.364089 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:34:14.364089 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:34:14.364089 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:34:14.364089 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:34:14.364089 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:34:14.364089 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:34:14.364089 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 10:34:14.364089 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 10:34:14.364089 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 10:34:14.364089 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 10:34:14.365143 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 10:34:14.365143 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 10:34:14.365143 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 10:34:14.365143 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 10:34:14.365143 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:34:14.365143 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:34:14.365143 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:34:14.365143 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:34:14.365143 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:34:14.365143 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:34:14.365143 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:34:14.365143 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application with Lock( Jul 27 10:34:14.365143 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:34:14.365143 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:34:14.365143 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:34:14.365143 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:34:14.366081 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:34:14.366081 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:34:14.366081 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:34:14.366081 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:34:14.366081 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 10:34:14.366081 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 10:34:14.366081 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 10:34:14.366081 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 10:34:14.366081 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 10:34:14.366081 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 10:34:14.366081 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 27 10:34:14.366081 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Jul 27 10:34:14.366712 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-eca7f4c8-b80e-412f-8274-560fd0e97b8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:34:14.366780 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 95/190] 10.222.0.36 () {56 vars in 1114 bytes} [Wed Jul 27 10:34:14 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 10:34:14.366780 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49ebd4d-599b-4849-a6c4-66190a5671a1 None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:14.387716 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:14.443999 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 96/191] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 10:34:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 10:34:14.444837 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca7f4c8-b80e-412f-8274-560fd0e97b8c None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:14.455636 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:14.481435 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49ebd4d-599b-4849-a6c4-66190a5671a1 None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:14.516541 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b305d8-d427-4e12-b799-24cf6e9e49fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:14.517182 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b305d8-d427-4e12-b799-24cf6e9e49fd None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:14.517461 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b305d8-d427-4e12-b799-24cf6e9e49fd None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:14.517731 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b305d8-d427-4e12-b799-24cf6e9e49fd None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:14.528231 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b305d8-d427-4e12-b799-24cf6e9e49fd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 10:34:14.533647 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application [None req-97b305d8-d427-4e12-b799-24cf6e9e49fd None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 27 10:34:14.533647 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:34:14.533647 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:34:14.533647 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:34:14.533647 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:34:14.533647 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:34:14.533647 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:34:14.533647 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:34:14.533647 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:34:14.533647 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:34:14.533647 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:34:14.533647 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:34:14.533647 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 10:34:14.533647 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 10:34:14.533647 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 10:34:14.533647 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 10:34:14.534732 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 10:34:14.534732 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 10:34:14.534732 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 10:34:14.534732 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 10:34:14.534732 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:34:14.534732 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:34:14.534732 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:34:14.534732 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:34:14.534732 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:34:14.534732 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:34:14.534732 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:34:14.534732 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application with Lock( Jul 27 10:34:14.534732 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:34:14.534732 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:34:14.534732 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:34:14.534732 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:34:14.535713 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:34:14.535713 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:34:14.535713 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:34:14.535713 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:34:14.535713 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 10:34:14.535713 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 10:34:14.535713 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 10:34:14.535713 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 10:34:14.535713 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 10:34:14.535713 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 10:34:14.535713 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 27 10:34:14.535713 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Jul 27 10:34:14.535713 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 96/192] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jul 27 10:34:14 2022] GET /identity/v3/roles/reader => generated 84 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 10:34:14.535713 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b305d8-d427-4e12-b799-24cf6e9e49fd None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:14.559456 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca7f4c8-b80e-412f-8274-560fd0e97b8c None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:14.595058 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f007a5c-bdfd-41d9-8873-a15febf5bf12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:14.595706 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f007a5c-bdfd-41d9-8873-a15febf5bf12 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:14.595980 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f007a5c-bdfd-41d9-8873-a15febf5bf12 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:14.596258 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f007a5c-bdfd-41d9-8873-a15febf5bf12 None admin] PATH_INFO: `/v3/users/533e1f1f20424aeda4aea04123be56d6` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:14.609244 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f007a5c-bdfd-41d9-8873-a15febf5bf12 None admin] RBAC: Authorizing `identity:get_user(user_id=533e1f1f20424aeda4aea04123be56d6)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:14.610736 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f007a5c-bdfd-41d9-8873-a15febf5bf12 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:14.612521 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 97/193] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Jul 27 10:34:14 2022] GET /identity/v3/users/533e1f1f20424aeda4aea04123be56d6 => generated 283 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:34:14.613115 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f007a5c-bdfd-41d9-8873-a15febf5bf12 None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:14.625364 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:14.637582 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b305d8-d427-4e12-b799-24cf6e9e49fd None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:14.670617 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cea9c33-4934-41bb-b8a6-e30ee7438d84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:14.671247 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cea9c33-4934-41bb-b8a6-e30ee7438d84 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:14.671530 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cea9c33-4934-41bb-b8a6-e30ee7438d84 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:14.671804 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cea9c33-4934-41bb-b8a6-e30ee7438d84 None admin] PATH_INFO: `/v3/roles` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:14.672522 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cea9c33-4934-41bb-b8a6-e30ee7438d84 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86827) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:34:14.672829 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cea9c33-4934-41bb-b8a6-e30ee7438d84 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:14.674213 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cea9c33-4934-41bb-b8a6-e30ee7438d84 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:14.675565 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:14.684920 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 97/194] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Jul 27 10:34:14 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:34:14.685706 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cea9c33-4934-41bb-b8a6-e30ee7438d84 None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:14.716092 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f007a5c-bdfd-41d9-8873-a15febf5bf12 None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:14.749106 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199f13fc-9bed-41a4-a444-9204065f8962 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:14.749822 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199f13fc-9bed-41a4-a444-9204065f8962 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:14.750100 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199f13fc-9bed-41a4-a444-9204065f8962 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:14.750372 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199f13fc-9bed-41a4-a444-9204065f8962 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:14.751530 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199f13fc-9bed-41a4-a444-9204065f8962 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:14.752965 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199f13fc-9bed-41a4-a444-9204065f8962 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:14.754548 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 98/195] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 10:34:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:34:14.755318 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199f13fc-9bed-41a4-a444-9204065f8962 None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:14.788293 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cea9c33-4934-41bb-b8a6-e30ee7438d84 None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:14.824047 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e5af84-69d0-4b09-a4bd-3d8cc8eb904c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:14.824594 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e5af84-69d0-4b09-a4bd-3d8cc8eb904c None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:14.824880 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e5af84-69d0-4b09-a4bd-3d8cc8eb904c None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:14.825181 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e5af84-69d0-4b09-a4bd-3d8cc8eb904c None admin] PATH_INFO: `/v3/roles` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:14.825900 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e5af84-69d0-4b09-a4bd-3d8cc8eb904c None admin] RBAC: Adding query filter params (name=reader) {{(pid=86827) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:34:14.826210 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e5af84-69d0-4b09-a4bd-3d8cc8eb904c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:14.827627 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e5af84-69d0-4b09-a4bd-3d8cc8eb904c None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:14.831063 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:14.836163 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 98/196] 10.222.0.36 () {56 vars in 1101 bytes} [Wed Jul 27 10:34:14 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:34:14.837038 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e5af84-69d0-4b09-a4bd-3d8cc8eb904c None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:14.855995 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199f13fc-9bed-41a4-a444-9204065f8962 None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:14.890645 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51cbd2f-7784-4461-9896-c65fb8eea605 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:14.891274 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51cbd2f-7784-4461-9896-c65fb8eea605 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:14.891560 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51cbd2f-7784-4461-9896-c65fb8eea605 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:14.891837 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51cbd2f-7784-4461-9896-c65fb8eea605 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:14.896035 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:14.939365 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e5af84-69d0-4b09-a4bd-3d8cc8eb904c None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:14.962728 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51cbd2f-7784-4461-9896-c65fb8eea605 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:14.964216 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51cbd2f-7784-4461-9896-c65fb8eea605 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:14.974322 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a43d45-300f-43eb-abcc-55c486b137d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:14.975012 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a43d45-300f-43eb-abcc-55c486b137d3 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:14.975296 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a43d45-300f-43eb-abcc-55c486b137d3 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:14.975572 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a43d45-300f-43eb-abcc-55c486b137d3 None admin] PATH_INFO: `/v3/users/533e1f1f20424aeda4aea04123be56d6` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:14.977295 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a43d45-300f-43eb-abcc-55c486b137d3 None admin] RBAC: Authorizing `identity:get_user(user_id=533e1f1f20424aeda4aea04123be56d6)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:14.978795 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a43d45-300f-43eb-abcc-55c486b137d3 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:14.980416 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 99/197] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Jul 27 10:34:14 2022] GET /identity/v3/users/533e1f1f20424aeda4aea04123be56d6 => generated 283 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:34:14.982758 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a43d45-300f-43eb-abcc-55c486b137d3 None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:15.045433 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:15.068124 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 99/198] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jul 27 10:34:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 10:34:15.068938 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51cbd2f-7784-4461-9896-c65fb8eea605 None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:15.083481 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a43d45-300f-43eb-abcc-55c486b137d3 None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:15.108919 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:15.118781 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1aacd6-373e-43e5-a0d3-60c6d73b1309 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:15.119401 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1aacd6-373e-43e5-a0d3-60c6d73b1309 None admin] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:15.119690 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1aacd6-373e-43e5-a0d3-60c6d73b1309 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:15.119971 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1aacd6-373e-43e5-a0d3-60c6d73b1309 None admin] PATH_INFO: `/v3/users` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:15.120656 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1aacd6-373e-43e5-a0d3-60c6d73b1309 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:15.122104 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1aacd6-373e-43e5-a0d3-60c6d73b1309 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:15.142516 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 100/199] 10.222.0.36 () {60 vars in 1132 bytes} [Wed Jul 27 10:34:14 2022] POST /identity/v3/users => generated 297 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 10:34:15.143754 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1aacd6-373e-43e5-a0d3-60c6d73b1309 None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:15.169883 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51cbd2f-7784-4461-9896-c65fb8eea605 None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:15.210190 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57db1f5b-f838-4ebd-9189-2c2d19b2f48a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:15.210893 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57db1f5b-f838-4ebd-9189-2c2d19b2f48a None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:15.211700 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57db1f5b-f838-4ebd-9189-2c2d19b2f48a None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:15.211978 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57db1f5b-f838-4ebd-9189-2c2d19b2f48a None admin] PATH_INFO: `/v3/users/aa21fc3e363e487d94f11e274b824cf8` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:15.213226 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57db1f5b-f838-4ebd-9189-2c2d19b2f48a None admin] RBAC: Authorizing `identity:get_user(user_id=aa21fc3e363e487d94f11e274b824cf8)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:15.214761 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57db1f5b-f838-4ebd-9189-2c2d19b2f48a None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:15.216435 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 100/200] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Jul 27 10:34:15 2022] GET /identity/v3/users/aa21fc3e363e487d94f11e274b824cf8 => generated 289 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:34:15.219651 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57db1f5b-f838-4ebd-9189-2c2d19b2f48a None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:15.278764 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1aacd6-373e-43e5-a0d3-60c6d73b1309 None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:15.279004 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:15.317232 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7562eb-5f57-4f14-9b33-dfb4944fb2e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:15.317971 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7562eb-5f57-4f14-9b33-dfb4944fb2e0 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:15.318258 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7562eb-5f57-4f14-9b33-dfb4944fb2e0 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:15.318535 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7562eb-5f57-4f14-9b33-dfb4944fb2e0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:15.344720 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57db1f5b-f838-4ebd-9189-2c2d19b2f48a None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:15.384034 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49763f4b-7555-40c3-82a1-a0ae1429de6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:15.384319 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:15.385074 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49763f4b-7555-40c3-82a1-a0ae1429de6e None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:15.385444 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49763f4b-7555-40c3-82a1-a0ae1429de6e None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:15.386041 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49763f4b-7555-40c3-82a1-a0ae1429de6e None admin] PATH_INFO: `/v3/projects/046cff0c48ee4df282871e4eda9cf939` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:15.386937 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49763f4b-7555-40c3-82a1-a0ae1429de6e None admin] RBAC: Authorizing `identity:get_project(project_id=046cff0c48ee4df282871e4eda9cf939)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:15.388402 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49763f4b-7555-40c3-82a1-a0ae1429de6e None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:15.392595 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 101/201] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Jul 27 10:34:15 2022] GET /identity/v3/projects/046cff0c48ee4df282871e4eda9cf939 => generated 302 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:34:15.392595 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49763f4b-7555-40c3-82a1-a0ae1429de6e None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:15.417014 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7562eb-5f57-4f14-9b33-dfb4944fb2e0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:15.418579 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7562eb-5f57-4f14-9b33-dfb4944fb2e0 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:15.521580 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49763f4b-7555-40c3-82a1-a0ae1429de6e None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:15.541350 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 101/202] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jul 27 10:34:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 10:34:15.542039 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:15.542197 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7562eb-5f57-4f14-9b33-dfb4944fb2e0 None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:15.569659 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb159ef-5727-4edd-bfd6-a22e76c1c735 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:15.570294 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb159ef-5727-4edd-bfd6-a22e76c1c735 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:15.571274 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb159ef-5727-4edd-bfd6-a22e76c1c735 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:15.571557 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb159ef-5727-4edd-bfd6-a22e76c1c735 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:15.648061 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:15.658469 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb159ef-5727-4edd-bfd6-a22e76c1c735 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:15.660307 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb159ef-5727-4edd-bfd6-a22e76c1c735 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:15.673085 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7562eb-5f57-4f14-9b33-dfb4944fb2e0 None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:15.717351 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bca764-e7a8-4d20-a6b5-81da714aab1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:15.717973 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bca764-e7a8-4d20-a6b5-81da714aab1f None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:15.718316 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bca764-e7a8-4d20-a6b5-81da714aab1f None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:15.718523 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bca764-e7a8-4d20-a6b5-81da714aab1f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:15.719366 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3bca764-e7a8-4d20-a6b5-81da714aab1f None admin] RBAC: Adding query filter params (role.id=21156560eb6b436bb567a40b4976f3f4, scope.project.id=046cff0c48ee4df282871e4eda9cf939, user.id=533e1f1f20424aeda4aea04123be56d6) {{(pid=86827) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:34:15.719680 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3bca764-e7a8-4d20-a6b5-81da714aab1f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:15.721137 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3bca764-e7a8-4d20-a6b5-81da714aab1f None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:15.734635 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 102/203] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Jul 27 10:34:15 2022] GET /identity/v3/role_assignments?user.id=533e1f1f20424aeda4aea04123be56d6&scope.project.id=046cff0c48ee4df282871e4eda9cf939&role.id=21156560eb6b436bb567a40b4976f3f4 => generated 259 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:34:15.734635 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bca764-e7a8-4d20-a6b5-81da714aab1f None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:15.819129 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 102/204] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jul 27 10:34:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 10:34:15.822384 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb159ef-5727-4edd-bfd6-a22e76c1c735 None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:15.837409 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bca764-e7a8-4d20-a6b5-81da714aab1f None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:15.846361 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:15.946629 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b80f10c-2dd5-4e3b-acab-52044ea659cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:15.947387 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b80f10c-2dd5-4e3b-acab-52044ea659cd None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:15.947590 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b80f10c-2dd5-4e3b-acab-52044ea659cd None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:15.947870 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b80f10c-2dd5-4e3b-acab-52044ea659cd None admin] PATH_INFO: `/v3/projects/046cff0c48ee4df282871e4eda9cf939` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:15.950518 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb159ef-5727-4edd-bfd6-a22e76c1c735 None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:15.955530 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b80f10c-2dd5-4e3b-acab-52044ea659cd None admin] RBAC: Authorizing `identity:get_project(project_id=046cff0c48ee4df282871e4eda9cf939)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:15.957045 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b80f10c-2dd5-4e3b-acab-52044ea659cd None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:15.959716 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 103/205] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Jul 27 10:34:15 2022] GET /identity/v3/projects/046cff0c48ee4df282871e4eda9cf939 => generated 302 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:34:15.967739 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b80f10c-2dd5-4e3b-acab-52044ea659cd None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:15.991182 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d5a605-e637-4acb-95b9-3d7c8560eb9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:15.992143 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d5a605-e637-4acb-95b9-3d7c8560eb9a None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:15.993809 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:15.994568 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d5a605-e637-4acb-95b9-3d7c8560eb9a None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:15.995053 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d5a605-e637-4acb-95b9-3d7c8560eb9a None admin] PATH_INFO: `/v3/projects/33cf2502c8694374bd447d6304e658c2` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:15.998878 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d5a605-e637-4acb-95b9-3d7c8560eb9a None admin] RBAC: Authorizing `identity:get_project(project_id=33cf2502c8694374bd447d6304e658c2)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:16.000533 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d5a605-e637-4acb-95b9-3d7c8560eb9a None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:16.004327 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 103/206] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Jul 27 10:34:15 2022] GET /identity/v3/projects/33cf2502c8694374bd447d6304e658c2 => generated 298 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:34:16.011835 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d5a605-e637-4acb-95b9-3d7c8560eb9a None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:16.111678 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b80f10c-2dd5-4e3b-acab-52044ea659cd None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:16.156520 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:16.180645 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d5a605-e637-4acb-95b9-3d7c8560eb9a None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:16.200788 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf00fb6-6192-4587-82fa-1e77e3189bc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:16.201289 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf00fb6-6192-4587-82fa-1e77e3189bc3 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:16.201568 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf00fb6-6192-4587-82fa-1e77e3189bc3 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:16.201847 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf00fb6-6192-4587-82fa-1e77e3189bc3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:16.202676 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf00fb6-6192-4587-82fa-1e77e3189bc3 None admin] RBAC: Adding query filter params (role.id=c3341f95596344309327efff99a20d5c, scope.project.id=046cff0c48ee4df282871e4eda9cf939, user.id=533e1f1f20424aeda4aea04123be56d6) {{(pid=86827) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:34:16.202994 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf00fb6-6192-4587-82fa-1e77e3189bc3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:16.204447 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf00fb6-6192-4587-82fa-1e77e3189bc3 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:16.217918 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 104/207] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Jul 27 10:34:15 2022] GET /identity/v3/role_assignments?user.id=533e1f1f20424aeda4aea04123be56d6&scope.project.id=046cff0c48ee4df282871e4eda9cf939&role.id=c3341f95596344309327efff99a20d5c => generated 259 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:34:16.220174 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb53e51c-e581-4b8e-a344-8831e69f4b56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:16.221043 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb53e51c-e581-4b8e-a344-8831e69f4b56 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:16.221519 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb53e51c-e581-4b8e-a344-8831e69f4b56 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:16.221986 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb53e51c-e581-4b8e-a344-8831e69f4b56 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:16.223044 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb53e51c-e581-4b8e-a344-8831e69f4b56 None admin] RBAC: Adding query filter params (role.id=80f67bfde4f5494e9c6d825aafb44e11, scope.project.id=33cf2502c8694374bd447d6304e658c2, user.id=aa21fc3e363e487d94f11e274b824cf8) {{(pid=86828) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:34:16.223583 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb53e51c-e581-4b8e-a344-8831e69f4b56 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:16.225257 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb53e51c-e581-4b8e-a344-8831e69f4b56 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:16.232977 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 104/208] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Jul 27 10:34:16 2022] GET /identity/v3/role_assignments?user.id=aa21fc3e363e487d94f11e274b824cf8&scope.project.id=33cf2502c8694374bd447d6304e658c2&role.id=80f67bfde4f5494e9c6d825aafb44e11 => generated 608 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:34:17.359163 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eeaacb2-c481-4ea6-b275-fba9d619d131 None None] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:17.359701 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eeaacb2-c481-4ea6-b275-fba9d619d131 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:17.359701 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eeaacb2-c481-4ea6-b275-fba9d619d131 None None] PATH_INFO: `/` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:17.361083 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 105/209] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 10:34:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 10:34:17.366335 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c249a473-3138-4720-85c9-9eca0c96693f None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:17.366741 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c249a473-3138-4720-85c9-9eca0c96693f None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:17.366959 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c249a473-3138-4720-85c9-9eca0c96693f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:17.386366 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-c249a473-3138-4720-85c9-9eca0c96693f None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:34:17.387583 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-c249a473-3138-4720-85c9-9eca0c96693f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:34:17.458337 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 105/210] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 10:34:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 10:34:17.482845 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eeaacb2-c481-4ea6-b275-fba9d619d131 None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:17.567391 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:17.569052 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af093a0e-673a-46e4-a6fb-0b503368da00 None None] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:17.569343 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af093a0e-673a-46e4-a6fb-0b503368da00 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:17.569631 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af093a0e-673a-46e4-a6fb-0b503368da00 None None] PATH_INFO: `/` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:17.571357 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 106/211] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 10:34:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 10:34:17.576852 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cc6465-aa05-4a4b-a173-ad16ab4c8f6a None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:17.577145 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cc6465-aa05-4a4b-a173-ad16ab4c8f6a None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:17.577426 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cc6465-aa05-4a4b-a173-ad16ab4c8f6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:17.598892 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eeaacb2-c481-4ea6-b275-fba9d619d131 None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:17.600933 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-a8cc6465-aa05-4a4b-a173-ad16ab4c8f6a None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:34:17.603615 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-a8cc6465-aa05-4a4b-a173-ad16ab4c8f6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:34:17.643463 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96276c4-f0be-4909-b348-dfe6592c0eed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:17.644494 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96276c4-f0be-4909-b348-dfe6592c0eed None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:17.644973 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96276c4-f0be-4909-b348-dfe6592c0eed None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:17.645443 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96276c4-f0be-4909-b348-dfe6592c0eed None admin] PATH_INFO: `/v3/roles/member` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:17.658993 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96276c4-f0be-4909-b348-dfe6592c0eed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 10:34:17.665753 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application [None req-d96276c4-f0be-4909-b348-dfe6592c0eed None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 27 10:34:17.665753 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:34:17.665753 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:34:17.665753 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:34:17.665753 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:34:17.665753 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:34:17.665753 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:34:17.665753 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:34:17.665753 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:34:17.665753 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:34:17.665753 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:34:17.665753 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:34:17.665753 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 10:34:17.665753 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 10:34:17.665753 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 10:34:17.665753 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 10:34:17.666628 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 10:34:17.666628 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 10:34:17.666628 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 10:34:17.666628 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 10:34:17.666628 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:34:17.666628 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:34:17.666628 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:34:17.666628 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:34:17.666628 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:34:17.666628 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:34:17.666628 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:34:17.666628 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application with Lock( Jul 27 10:34:17.666628 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:34:17.666628 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:34:17.666628 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:34:17.666628 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:34:17.667422 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:34:17.667422 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:34:17.667422 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:34:17.667422 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:34:17.667422 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 10:34:17.667422 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 10:34:17.667422 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 10:34:17.667422 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 10:34:17.667422 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 10:34:17.667422 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 10:34:17.667422 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 27 10:34:17.667422 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Jul 27 10:34:17.670022 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 106/212] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jul 27 10:34:17 2022] GET /identity/v3/roles/member => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 10:34:17.676713 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96276c4-f0be-4909-b348-dfe6592c0eed None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:17.696557 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 107/213] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 10:34:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 10:34:17.721140 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cc6465-aa05-4a4b-a173-ad16ab4c8f6a None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:17.730064 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:17.810952 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96276c4-f0be-4909-b348-dfe6592c0eed None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:17.832188 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cc6465-aa05-4a4b-a173-ad16ab4c8f6a None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:17.860715 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03462b5-5578-4397-853c-326af58a77b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:17.861991 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03462b5-5578-4397-853c-326af58a77b1 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:17.862498 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03462b5-5578-4397-853c-326af58a77b1 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:17.863289 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03462b5-5578-4397-853c-326af58a77b1 None admin] PATH_INFO: `/v3/roles` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:17.864908 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03462b5-5578-4397-853c-326af58a77b1 None admin] RBAC: Adding query filter params (name=member) {{(pid=86827) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:34:17.865436 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03462b5-5578-4397-853c-326af58a77b1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:17.868089 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03462b5-5578-4397-853c-326af58a77b1 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:17.871783 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d341b6-bfcc-4ecd-827f-6de2e1ccb4e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:17.872435 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d341b6-bfcc-4ecd-827f-6de2e1ccb4e1 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:17.872719 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d341b6-bfcc-4ecd-827f-6de2e1ccb4e1 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:17.872992 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d341b6-bfcc-4ecd-827f-6de2e1ccb4e1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:17.874162 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d341b6-bfcc-4ecd-827f-6de2e1ccb4e1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:17.875326 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:17.875634 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d341b6-bfcc-4ecd-827f-6de2e1ccb4e1 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:17.881761 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 107/214] 10.222.0.36 () {56 vars in 1101 bytes} [Wed Jul 27 10:34:17 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 10:34:17.883410 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03462b5-5578-4397-853c-326af58a77b1 None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:17.884959 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 108/215] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 10:34:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:34:17.888597 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d341b6-bfcc-4ecd-827f-6de2e1ccb4e1 None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:17.984247 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:18.017501 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03462b5-5578-4397-853c-326af58a77b1 None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:18.025306 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d341b6-bfcc-4ecd-827f-6de2e1ccb4e1 None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:18.053949 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afaff251-e5ba-4d55-96f5-bc2145d7d624 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:18.054593 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afaff251-e5ba-4d55-96f5-bc2145d7d624 None admin] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:18.054951 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afaff251-e5ba-4d55-96f5-bc2145d7d624 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:18.055234 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afaff251-e5ba-4d55-96f5-bc2145d7d624 None admin] PATH_INFO: `/v3/users` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:18.055941 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afaff251-e5ba-4d55-96f5-bc2145d7d624 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:18.057407 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afaff251-e5ba-4d55-96f5-bc2145d7d624 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:18.072222 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bcf487-2c0b-499d-92f5-0828b4e1cacd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:18.073254 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bcf487-2c0b-499d-92f5-0828b4e1cacd None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:18.073538 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bcf487-2c0b-499d-92f5-0828b4e1cacd None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:18.073935 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bcf487-2c0b-499d-92f5-0828b4e1cacd None admin] PATH_INFO: `/v3/users/73fa0cd345004953a63f05de2ddc4ef4` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:18.081205 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 108/216] 10.222.0.36 () {60 vars in 1132 bytes} [Wed Jul 27 10:34:17 2022] POST /identity/v3/users => generated 297 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 10:34:18.088775 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ce1cca-a757-4533-a6db-cd7fbd2b3a82 None None] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:18.089350 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ce1cca-a757-4533-a6db-cd7fbd2b3a82 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:18.091326 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ce1cca-a757-4533-a6db-cd7fbd2b3a82 None None] PATH_INFO: `/` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:18.092849 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 109/217] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 10:34:18 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 10:34:18.101371 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e09cddd-0da9-4c03-868a-35cc7051f62d None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:18.101765 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e09cddd-0da9-4c03-868a-35cc7051f62d None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:18.102019 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e09cddd-0da9-4c03-868a-35cc7051f62d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:18.104815 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bcf487-2c0b-499d-92f5-0828b4e1cacd None admin] RBAC: Authorizing `identity:get_user(user_id=73fa0cd345004953a63f05de2ddc4ef4)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:18.107348 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bcf487-2c0b-499d-92f5-0828b4e1cacd None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:18.110376 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 109/218] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Jul 27 10:34:17 2022] GET /identity/v3/users/73fa0cd345004953a63f05de2ddc4ef4 => generated 297 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:34:18.115574 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bcf487-2c0b-499d-92f5-0828b4e1cacd None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:18.123538 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-1e09cddd-0da9-4c03-868a-35cc7051f62d None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:34:18.124819 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-1e09cddd-0da9-4c03-868a-35cc7051f62d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:34:18.196271 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:18.217782 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 110/219] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 10:34:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 10:34:18.240318 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e09cddd-0da9-4c03-868a-35cc7051f62d None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:18.246662 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bcf487-2c0b-499d-92f5-0828b4e1cacd None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:18.284191 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:18.291463 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9c2926-fddc-4aab-a07f-9d5fa570feb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:18.291463 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9c2926-fddc-4aab-a07f-9d5fa570feb9 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:18.291688 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9c2926-fddc-4aab-a07f-9d5fa570feb9 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:18.292331 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9c2926-fddc-4aab-a07f-9d5fa570feb9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:18.380675 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e09cddd-0da9-4c03-868a-35cc7051f62d None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:18.390986 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9c2926-fddc-4aab-a07f-9d5fa570feb9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:18.392659 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9c2926-fddc-4aab-a07f-9d5fa570feb9 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:18.437429 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c89c84b-73db-4417-9b83-8180fabdfff9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:18.437704 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c89c84b-73db-4417-9b83-8180fabdfff9 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:18.437816 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c89c84b-73db-4417-9b83-8180fabdfff9 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:18.438108 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c89c84b-73db-4417-9b83-8180fabdfff9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:18.451721 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c89c84b-73db-4417-9b83-8180fabdfff9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 10:34:18.457179 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application [None req-4c89c84b-73db-4417-9b83-8180fabdfff9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 10:34:18.457179 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:34:18.457179 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:34:18.457179 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:34:18.457179 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:34:18.457179 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:34:18.457179 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:34:18.457179 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:34:18.457179 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:34:18.457179 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:34:18.457179 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:34:18.457179 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:34:18.457179 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 10:34:18.457179 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 10:34:18.457179 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 10:34:18.457179 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 10:34:18.458830 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 10:34:18.458830 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 10:34:18.458830 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 10:34:18.458830 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 10:34:18.458830 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:34:18.458830 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:34:18.458830 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:34:18.458830 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:34:18.458830 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:34:18.458830 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:34:18.458830 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:34:18.458830 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application with Lock( Jul 27 10:34:18.458830 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:34:18.458830 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:34:18.458830 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:34:18.458830 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:34:18.460113 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:34:18.460113 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:34:18.460113 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:34:18.460113 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:34:18.460113 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 10:34:18.460113 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 10:34:18.460113 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 10:34:18.460113 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 10:34:18.460113 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 10:34:18.460113 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 10:34:18.460113 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 10:34:18.460113 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Jul 27 10:34:18.460113 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 111/220] 10.222.0.36 () {56 vars in 1096 bytes} [Wed Jul 27 10:34:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 10:34:18.460113 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe1a3fb-fcf5-45c7-b2fc-64e8781a488b None None] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:18.461321 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe1a3fb-fcf5-45c7-b2fc-64e8781a488b None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:18.461321 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe1a3fb-fcf5-45c7-b2fc-64e8781a488b None None] PATH_INFO: `/` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:18.466521 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 112/221] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 10:34:18 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 10:34:18.472560 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f828a64-33bc-47b3-9817-00a615e4848d None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:18.473133 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f828a64-33bc-47b3-9817-00a615e4848d None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:18.473420 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f828a64-33bc-47b3-9817-00a615e4848d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:18.499982 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-3f828a64-33bc-47b3-9817-00a615e4848d None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:34:18.501454 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-3f828a64-33bc-47b3-9817-00a615e4848d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:34:18.523572 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 110/222] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jul 27 10:34:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 10:34:18.524526 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9c2926-fddc-4aab-a07f-9d5fa570feb9 None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:18.556287 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:18.571473 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:18.590309 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 113/223] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 10:34:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 10:34:18.594912 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f828a64-33bc-47b3-9817-00a615e4848d None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:18.662875 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9c2926-fddc-4aab-a07f-9d5fa570feb9 None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:18.701931 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff239da8-766c-4dcb-8246-fc92dedec06a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:18.702469 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff239da8-766c-4dcb-8246-fc92dedec06a None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:18.702774 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff239da8-766c-4dcb-8246-fc92dedec06a None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:18.703048 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff239da8-766c-4dcb-8246-fc92dedec06a None admin] PATH_INFO: `/v3/roles` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:18.703764 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff239da8-766c-4dcb-8246-fc92dedec06a None admin] RBAC: Adding query filter params (name=admin) {{(pid=86828) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:34:18.704065 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff239da8-766c-4dcb-8246-fc92dedec06a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:18.705551 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff239da8-766c-4dcb-8246-fc92dedec06a None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:18.715352 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 111/224] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jul 27 10:34:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:34:18.716349 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff239da8-766c-4dcb-8246-fc92dedec06a None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:18.719358 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f828a64-33bc-47b3-9817-00a615e4848d None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:18.766419 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3573d793-2506-4b38-898c-4d9cea39dd96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:18.767247 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3573d793-2506-4b38-898c-4d9cea39dd96 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:18.767659 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3573d793-2506-4b38-898c-4d9cea39dd96 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:18.767944 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3573d793-2506-4b38-898c-4d9cea39dd96 None admin] PATH_INFO: `/v3/projects/046cff0c48ee4df282871e4eda9cf939` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:18.769469 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3573d793-2506-4b38-898c-4d9cea39dd96 None admin] RBAC: Authorizing `identity:get_project(project_id=046cff0c48ee4df282871e4eda9cf939)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:18.771324 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3573d793-2506-4b38-898c-4d9cea39dd96 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:18.773946 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 114/225] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Jul 27 10:34:18 2022] GET /identity/v3/projects/046cff0c48ee4df282871e4eda9cf939 => generated 302 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:34:18.774761 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3573d793-2506-4b38-898c-4d9cea39dd96 None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:18.833451 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:18.842031 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff239da8-766c-4dcb-8246-fc92dedec06a None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:18.848973 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:18.879231 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3163370-7d6e-4a4c-93f4-bed540f138ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:18.880042 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3163370-7d6e-4a4c-93f4-bed540f138ae None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:18.880423 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3163370-7d6e-4a4c-93f4-bed540f138ae None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:18.880800 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3163370-7d6e-4a4c-93f4-bed540f138ae None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:18.893528 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3163370-7d6e-4a4c-93f4-bed540f138ae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 10:34:18.900172 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application [None req-a3163370-7d6e-4a4c-93f4-bed540f138ae None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 27 10:34:18.900172 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:34:18.900172 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:34:18.900172 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:34:18.900172 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:34:18.900172 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:34:18.900172 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:34:18.900172 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:34:18.900172 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:34:18.900172 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:34:18.900172 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:34:18.900172 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:34:18.900172 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 10:34:18.900172 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 10:34:18.900172 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 10:34:18.900172 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 10:34:18.901271 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 10:34:18.901271 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 10:34:18.901271 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 10:34:18.901271 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 10:34:18.901271 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:34:18.901271 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:34:18.901271 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:34:18.901271 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:34:18.901271 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:34:18.901271 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:34:18.901271 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:34:18.901271 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application with Lock( Jul 27 10:34:18.901271 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:34:18.901271 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:34:18.901271 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:34:18.901271 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:34:18.902234 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:34:18.902234 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:34:18.902234 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:34:18.902234 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:34:18.902234 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 10:34:18.902234 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 10:34:18.902234 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 10:34:18.902234 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 10:34:18.902234 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 10:34:18.902234 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 10:34:18.902234 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 27 10:34:18.902234 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Jul 27 10:34:18.904243 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 112/226] 10.222.0.36 () {56 vars in 1114 bytes} [Wed Jul 27 10:34:18 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 10:34:18.905164 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3163370-7d6e-4a4c-93f4-bed540f138ae None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:18.925523 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3573d793-2506-4b38-898c-4d9cea39dd96 None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:18.975406 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b23e3e-4ee5-485c-a204-763a8039175c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:18.976329 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b23e3e-4ee5-485c-a204-763a8039175c None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:18.976806 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b23e3e-4ee5-485c-a204-763a8039175c None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:18.977289 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b23e3e-4ee5-485c-a204-763a8039175c None admin] PATH_INFO: `/v3/users/533e1f1f20424aeda4aea04123be56d6` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:18.978886 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b23e3e-4ee5-485c-a204-763a8039175c None admin] RBAC: Authorizing `identity:get_user(user_id=533e1f1f20424aeda4aea04123be56d6)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:18.980604 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b23e3e-4ee5-485c-a204-763a8039175c None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:18.982383 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 115/227] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Jul 27 10:34:18 2022] GET /identity/v3/users/533e1f1f20424aeda4aea04123be56d6 => generated 283 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:34:18.983367 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b23e3e-4ee5-485c-a204-763a8039175c None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:19.013675 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3163370-7d6e-4a4c-93f4-bed540f138ae None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:19.048297 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e154f927-8dea-4376-a996-8a1823685b7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:19.049080 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e154f927-8dea-4376-a996-8a1823685b7f None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:19.049471 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e154f927-8dea-4376-a996-8a1823685b7f None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:19.049853 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e154f927-8dea-4376-a996-8a1823685b7f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:19.050935 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e154f927-8dea-4376-a996-8a1823685b7f None admin] RBAC: Adding query filter params (role.id=4c9fc0b3baed441da2126be41cc6b04a, scope.project.id=046cff0c48ee4df282871e4eda9cf939, user.id=73fa0cd345004953a63f05de2ddc4ef4) {{(pid=86828) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:34:19.051263 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e154f927-8dea-4376-a996-8a1823685b7f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:19.052819 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e154f927-8dea-4376-a996-8a1823685b7f None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:19.059196 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 113/228] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Jul 27 10:34:18 2022] GET /identity/v3/role_assignments?user.id=73fa0cd345004953a63f05de2ddc4ef4&scope.project.id=046cff0c48ee4df282871e4eda9cf939&role.id=4c9fc0b3baed441da2126be41cc6b04a => generated 259 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:34:19.067744 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e154f927-8dea-4376-a996-8a1823685b7f None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:19.096326 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:19.100284 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b23e3e-4ee5-485c-a204-763a8039175c None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:19.129681 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:19.178191 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fa61da-e0ec-4088-9576-04ae55622d0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:19.179247 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fa61da-e0ec-4088-9576-04ae55622d0d None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:19.180292 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fa61da-e0ec-4088-9576-04ae55622d0d None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:19.180765 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fa61da-e0ec-4088-9576-04ae55622d0d None admin] PATH_INFO: `/v3/roles` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:19.181690 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8fa61da-e0ec-4088-9576-04ae55622d0d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86827) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:34:19.182192 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8fa61da-e0ec-4088-9576-04ae55622d0d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:19.183978 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8fa61da-e0ec-4088-9576-04ae55622d0d None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:19.193817 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 116/229] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Jul 27 10:34:18 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:34:19.200436 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fa61da-e0ec-4088-9576-04ae55622d0d None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:19.290890 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e154f927-8dea-4376-a996-8a1823685b7f None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:19.345447 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fa61da-e0ec-4088-9576-04ae55622d0d None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:19.358528 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b412935-c69a-4d9d-89ec-cdbb373c4e69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:19.359212 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b412935-c69a-4d9d-89ec-cdbb373c4e69 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:19.359492 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b412935-c69a-4d9d-89ec-cdbb373c4e69 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:19.359762 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b412935-c69a-4d9d-89ec-cdbb373c4e69 None admin] PATH_INFO: `/v3/projects/046cff0c48ee4df282871e4eda9cf939` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:19.362558 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b412935-c69a-4d9d-89ec-cdbb373c4e69 None admin] RBAC: Authorizing `identity:get_project(project_id=046cff0c48ee4df282871e4eda9cf939)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:19.364145 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b412935-c69a-4d9d-89ec-cdbb373c4e69 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:19.366347 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 114/230] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Jul 27 10:34:19 2022] GET /identity/v3/projects/046cff0c48ee4df282871e4eda9cf939 => generated 302 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:34:19.374215 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b412935-c69a-4d9d-89ec-cdbb373c4e69 None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:19.419266 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81839c52-8842-4c5e-9555-50d2b6ed9409 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:19.419943 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81839c52-8842-4c5e-9555-50d2b6ed9409 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:19.420198 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81839c52-8842-4c5e-9555-50d2b6ed9409 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:19.420472 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81839c52-8842-4c5e-9555-50d2b6ed9409 None admin] PATH_INFO: `/v3/users/533e1f1f20424aeda4aea04123be56d6` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:19.422387 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81839c52-8842-4c5e-9555-50d2b6ed9409 None admin] RBAC: Authorizing `identity:get_user(user_id=533e1f1f20424aeda4aea04123be56d6)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:19.423872 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81839c52-8842-4c5e-9555-50d2b6ed9409 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:19.425652 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 117/231] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Jul 27 10:34:19 2022] GET /identity/v3/users/533e1f1f20424aeda4aea04123be56d6 => generated 283 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:34:19.434834 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81839c52-8842-4c5e-9555-50d2b6ed9409 None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:19.526995 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:19.529089 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:19.572716 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b412935-c69a-4d9d-89ec-cdbb373c4e69 None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:19.587535 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81839c52-8842-4c5e-9555-50d2b6ed9409 None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:19.623412 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e39b63-9bfd-439e-88c3-c1eec0232484 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:19.626633 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e39b63-9bfd-439e-88c3-c1eec0232484 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:19.626633 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e39b63-9bfd-439e-88c3-c1eec0232484 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:19.626633 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e39b63-9bfd-439e-88c3-c1eec0232484 None admin] PATH_INFO: `/v3/projects/046cff0c48ee4df282871e4eda9cf939` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:19.626633 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e39b63-9bfd-439e-88c3-c1eec0232484 None admin] RBAC: Authorizing `identity:get_project(project_id=046cff0c48ee4df282871e4eda9cf939)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:19.627267 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e39b63-9bfd-439e-88c3-c1eec0232484 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:19.629064 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 118/232] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Jul 27 10:34:19 2022] GET /identity/v3/projects/046cff0c48ee4df282871e4eda9cf939 => generated 302 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:34:19.635021 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e39b63-9bfd-439e-88c3-c1eec0232484 None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:19.656184 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98980ee-1040-4a04-909a-c36495bdda54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:19.657178 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98980ee-1040-4a04-909a-c36495bdda54 None admin] REQUEST_METHOD: `PUT` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:19.657685 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98980ee-1040-4a04-909a-c36495bdda54 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:19.658173 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98980ee-1040-4a04-909a-c36495bdda54 None admin] PATH_INFO: `/v3/projects/046cff0c48ee4df282871e4eda9cf939/users/533e1f1f20424aeda4aea04123be56d6/roles/21156560eb6b436bb567a40b4976f3f4` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:19.660514 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98980ee-1040-4a04-909a-c36495bdda54 None admin] RBAC: Authorizing `identity:create_grant(project_id=046cff0c48ee4df282871e4eda9cf939, user_id=533e1f1f20424aeda4aea04123be56d6, role_id=21156560eb6b436bb567a40b4976f3f4)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:19.662237 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98980ee-1040-4a04-909a-c36495bdda54 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:19.705196 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 115/233] 10.222.0.36 () {58 vars in 1439 bytes} [Wed Jul 27 10:34:19 2022] PUT /identity/v3/projects/046cff0c48ee4df282871e4eda9cf939/users/533e1f1f20424aeda4aea04123be56d6/roles/21156560eb6b436bb567a40b4976f3f4 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 10:34:19.740062 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e39b63-9bfd-439e-88c3-c1eec0232484 None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:19.752221 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:19.778116 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30e3dd3-0747-4fc5-b304-9153f2be76e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:19.778837 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30e3dd3-0747-4fc5-b304-9153f2be76e4 None admin] REQUEST_METHOD: `PUT` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:19.779123 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30e3dd3-0747-4fc5-b304-9153f2be76e4 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:19.779402 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30e3dd3-0747-4fc5-b304-9153f2be76e4 None admin] PATH_INFO: `/v3/projects/046cff0c48ee4df282871e4eda9cf939/users/533e1f1f20424aeda4aea04123be56d6/roles/c3341f95596344309327efff99a20d5c` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:19.781653 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30e3dd3-0747-4fc5-b304-9153f2be76e4 None admin] RBAC: Authorizing `identity:create_grant(project_id=046cff0c48ee4df282871e4eda9cf939, user_id=533e1f1f20424aeda4aea04123be56d6, role_id=c3341f95596344309327efff99a20d5c)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:19.783137 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30e3dd3-0747-4fc5-b304-9153f2be76e4 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:19.805591 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 119/234] 10.222.0.36 () {58 vars in 1439 bytes} [Wed Jul 27 10:34:19 2022] PUT /identity/v3/projects/046cff0c48ee4df282871e4eda9cf939/users/533e1f1f20424aeda4aea04123be56d6/roles/c3341f95596344309327efff99a20d5c => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:34:20.481674 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fb2ef1-c00c-4f70-9110-da0a75833df8 None None] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:20.482072 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fb2ef1-c00c-4f70-9110-da0a75833df8 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:20.482247 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fb2ef1-c00c-4f70-9110-da0a75833df8 None None] PATH_INFO: `/` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:20.483649 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 116/235] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 10:34:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 10:34:20.488874 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270e9494-a432-48c4-8d0c-322cf1512ae5 None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:20.489304 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270e9494-a432-48c4-8d0c-322cf1512ae5 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:20.489485 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270e9494-a432-48c4-8d0c-322cf1512ae5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:20.515087 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-270e9494-a432-48c4-8d0c-322cf1512ae5 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:34:20.516287 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-270e9494-a432-48c4-8d0c-322cf1512ae5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:34:20.601419 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 120/236] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 10:34:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 10:34:20.637691 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fb2ef1-c00c-4f70-9110-da0a75833df8 None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:20.677954 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed17eebb-1b95-4401-b049-b34a67492547 None None] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:20.678257 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed17eebb-1b95-4401-b049-b34a67492547 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:20.678515 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed17eebb-1b95-4401-b049-b34a67492547 None None] PATH_INFO: `/` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:20.680453 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 121/237] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 10:34:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 10:34:20.690992 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213d2872-11af-4429-858f-d546a8a3aa31 None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:20.691275 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213d2872-11af-4429-858f-d546a8a3aa31 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:20.691543 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213d2872-11af-4429-858f-d546a8a3aa31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:20.721162 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-213d2872-11af-4429-858f-d546a8a3aa31 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:34:20.722361 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-213d2872-11af-4429-858f-d546a8a3aa31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:34:20.792945 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fb2ef1-c00c-4f70-9110-da0a75833df8 None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:20.828623 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfe1f36-6806-4f3f-9df4-7d74d12385fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:20.829287 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfe1f36-6806-4f3f-9df4-7d74d12385fc None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:20.829574 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfe1f36-6806-4f3f-9df4-7d74d12385fc None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:20.829854 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfe1f36-6806-4f3f-9df4-7d74d12385fc None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:20.840298 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfe1f36-6806-4f3f-9df4-7d74d12385fc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 10:34:20.845660 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application [None req-dcfe1f36-6806-4f3f-9df4-7d74d12385fc None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 27 10:34:20.845660 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:34:20.845660 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:34:20.845660 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:34:20.845660 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:34:20.845660 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:34:20.845660 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:34:20.845660 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:34:20.845660 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:34:20.845660 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:34:20.845660 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:34:20.845660 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:34:20.845660 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 10:34:20.845660 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 10:34:20.845660 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 10:34:20.845660 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 10:34:20.847682 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 10:34:20.847682 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 10:34:20.847682 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 10:34:20.847682 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 10:34:20.847682 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:34:20.847682 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:34:20.847682 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:34:20.847682 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:34:20.847682 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:34:20.847682 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:34:20.847682 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:34:20.847682 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application with Lock( Jul 27 10:34:20.847682 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:34:20.847682 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:34:20.847682 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:34:20.847682 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:34:20.849315 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:34:20.849315 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:34:20.849315 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:34:20.849315 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:34:20.849315 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 10:34:20.849315 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 10:34:20.849315 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 10:34:20.849315 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 10:34:20.849315 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 10:34:20.849315 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 10:34:20.849315 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 27 10:34:20.849315 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Jul 27 10:34:20.849315 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 117/238] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jul 27 10:34:20 2022] GET /identity/v3/roles/reader => generated 84 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 10:34:20.854656 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 122/239] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 10:34:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 27 10:34:20.855701 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfe1f36-6806-4f3f-9df4-7d74d12385fc None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:20.864110 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:20.910475 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213d2872-11af-4429-858f-d546a8a3aa31 None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:20.968812 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfe1f36-6806-4f3f-9df4-7d74d12385fc None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:20.982207 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:21.005716 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d710bcc2-84df-416f-a6c3-8c8c4743b256 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:21.005716 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d710bcc2-84df-416f-a6c3-8c8c4743b256 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:21.005716 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d710bcc2-84df-416f-a6c3-8c8c4743b256 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:21.005716 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d710bcc2-84df-416f-a6c3-8c8c4743b256 None admin] PATH_INFO: `/v3/roles` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:21.006117 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d710bcc2-84df-416f-a6c3-8c8c4743b256 None admin] RBAC: Adding query filter params (name=reader) {{(pid=86828) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:34:21.006408 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d710bcc2-84df-416f-a6c3-8c8c4743b256 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:21.007708 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d710bcc2-84df-416f-a6c3-8c8c4743b256 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:21.016781 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 118/240] 10.222.0.36 () {56 vars in 1101 bytes} [Wed Jul 27 10:34:20 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:34:21.023677 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d710bcc2-84df-416f-a6c3-8c8c4743b256 None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:21.129631 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:21.143523 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213d2872-11af-4429-858f-d546a8a3aa31 None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:21.187498 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eed8ac6-6c85-44ea-a83f-6382cac4bb43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:21.188189 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eed8ac6-6c85-44ea-a83f-6382cac4bb43 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:21.188474 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eed8ac6-6c85-44ea-a83f-6382cac4bb43 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:21.188772 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eed8ac6-6c85-44ea-a83f-6382cac4bb43 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:21.190380 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eed8ac6-6c85-44ea-a83f-6382cac4bb43 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:21.191932 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eed8ac6-6c85-44ea-a83f-6382cac4bb43 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:21.193920 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 123/241] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 10:34:20 2022] GET /identity/v3/domains/default => generated 199 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 10:34:21.200789 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eed8ac6-6c85-44ea-a83f-6382cac4bb43 None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:21.224766 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d710bcc2-84df-416f-a6c3-8c8c4743b256 None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:21.314020 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:21.315606 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d90a7b-eec1-4519-b172-8db4805fd31f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:21.316709 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d90a7b-eec1-4519-b172-8db4805fd31f None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:21.317193 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d90a7b-eec1-4519-b172-8db4805fd31f None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:21.317656 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d90a7b-eec1-4519-b172-8db4805fd31f None admin] PATH_INFO: `/v3/users/9a8d5697b327417aa265bb38fb008a75` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:21.317993 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eed8ac6-6c85-44ea-a83f-6382cac4bb43 None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:21.339974 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d90a7b-eec1-4519-b172-8db4805fd31f None admin] RBAC: Authorizing `identity:get_user(user_id=9a8d5697b327417aa265bb38fb008a75)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:21.341498 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d90a7b-eec1-4519-b172-8db4805fd31f None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:21.343508 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 119/242] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Jul 27 10:34:21 2022] GET /identity/v3/users/9a8d5697b327417aa265bb38fb008a75 => generated 297 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:34:21.350235 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d90a7b-eec1-4519-b172-8db4805fd31f None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:21.379924 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3325452f-1ad8-4fe6-ae0a-5555a6d3c6c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:21.380484 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3325452f-1ad8-4fe6-ae0a-5555a6d3c6c0 None admin] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:21.380770 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3325452f-1ad8-4fe6-ae0a-5555a6d3c6c0 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:21.381047 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3325452f-1ad8-4fe6-ae0a-5555a6d3c6c0 None admin] PATH_INFO: `/v3/users` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:21.381745 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3325452f-1ad8-4fe6-ae0a-5555a6d3c6c0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:21.383262 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3325452f-1ad8-4fe6-ae0a-5555a6d3c6c0 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:21.430292 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 124/243] 10.222.0.36 () {60 vars in 1132 bytes} [Wed Jul 27 10:34:21 2022] POST /identity/v3/users => generated 293 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 10:34:21.493732 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d90a7b-eec1-4519-b172-8db4805fd31f None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:21.522050 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:21.532314 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62218905-d340-497f-b393-7bc4c2b2513e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:21.532969 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62218905-d340-497f-b393-7bc4c2b2513e None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:21.533250 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62218905-d340-497f-b393-7bc4c2b2513e None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:21.533530 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62218905-d340-497f-b393-7bc4c2b2513e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:21.603900 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6529931d-42f7-43bb-bf98-00a11b0e9aeb None None] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:21.604415 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6529931d-42f7-43bb-bf98-00a11b0e9aeb None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:21.604488 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6529931d-42f7-43bb-bf98-00a11b0e9aeb None None] PATH_INFO: `/` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:21.605863 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 125/244] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 10:34:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 10:34:21.611521 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5fd421-2861-45aa-ba9b-868d3ac41a2d None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:21.611824 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5fd421-2861-45aa-ba9b-868d3ac41a2d None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:21.612072 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5fd421-2861-45aa-ba9b-868d3ac41a2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:21.614488 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62218905-d340-497f-b393-7bc4c2b2513e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:21.616349 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62218905-d340-497f-b393-7bc4c2b2513e None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:21.631319 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-4c5fd421-2861-45aa-ba9b-868d3ac41a2d None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:34:21.632435 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-4c5fd421-2861-45aa-ba9b-868d3ac41a2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:34:21.713946 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 126/245] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 10:34:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 27 10:34:21.737541 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5fd421-2861-45aa-ba9b-868d3ac41a2d None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:21.744649 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 120/246] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jul 27 10:34:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 10:34:21.750812 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62218905-d340-497f-b393-7bc4c2b2513e None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:21.787313 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:21.844887 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:21.865538 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5fd421-2861-45aa-ba9b-868d3ac41a2d None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:21.887265 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62218905-d340-497f-b393-7bc4c2b2513e None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:21.905485 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb09c34c-1c5f-4a61-9199-c5d67b0fc1fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:21.906139 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb09c34c-1c5f-4a61-9199-c5d67b0fc1fc None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:21.906677 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb09c34c-1c5f-4a61-9199-c5d67b0fc1fc None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:21.906677 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb09c34c-1c5f-4a61-9199-c5d67b0fc1fc None admin] PATH_INFO: `/v3/roles/member` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:21.918425 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb09c34c-1c5f-4a61-9199-c5d67b0fc1fc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 10:34:21.923385 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application [None req-bb09c34c-1c5f-4a61-9199-c5d67b0fc1fc None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 27 10:34:21.923385 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:34:21.923385 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:34:21.923385 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:34:21.923385 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:34:21.923385 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:34:21.923385 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:34:21.923385 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:34:21.923385 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:34:21.923385 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:34:21.923385 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:34:21.923385 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:34:21.923385 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 10:34:21.923385 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 10:34:21.923385 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 10:34:21.923385 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 10:34:21.924244 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 10:34:21.924244 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 10:34:21.924244 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 10:34:21.924244 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 10:34:21.924244 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:34:21.924244 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:34:21.924244 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:34:21.924244 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:34:21.924244 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:34:21.924244 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:34:21.924244 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:34:21.924244 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application with Lock( Jul 27 10:34:21.924244 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:34:21.924244 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:34:21.924244 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:34:21.924244 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:34:21.925123 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:34:21.925123 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:34:21.925123 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:34:21.925123 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:34:21.925123 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 10:34:21.925123 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 10:34:21.925123 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 10:34:21.925123 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 10:34:21.925123 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 10:34:21.925123 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 10:34:21.925123 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 27 10:34:21.925123 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Jul 27 10:34:21.929377 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 127/247] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jul 27 10:34:21 2022] GET /identity/v3/roles/member => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 10:34:21.931144 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb09c34c-1c5f-4a61-9199-c5d67b0fc1fc None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:21.934144 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bb510d-8d7e-4936-bf9e-8e006bd07ef0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:21.935260 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bb510d-8d7e-4936-bf9e-8e006bd07ef0 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:21.936571 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bb510d-8d7e-4936-bf9e-8e006bd07ef0 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:21.937086 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bb510d-8d7e-4936-bf9e-8e006bd07ef0 None admin] PATH_INFO: `/v3/projects/046cff0c48ee4df282871e4eda9cf939` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:21.938632 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01bb510d-8d7e-4936-bf9e-8e006bd07ef0 None admin] RBAC: Authorizing `identity:get_project(project_id=046cff0c48ee4df282871e4eda9cf939)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:21.940318 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01bb510d-8d7e-4936-bf9e-8e006bd07ef0 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:21.943467 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 121/248] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Jul 27 10:34:21 2022] GET /identity/v3/projects/046cff0c48ee4df282871e4eda9cf939 => generated 302 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:34:21.949503 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bb510d-8d7e-4936-bf9e-8e006bd07ef0 None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:22.057281 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb09c34c-1c5f-4a61-9199-c5d67b0fc1fc None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:22.072699 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:22.081994 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bb510d-8d7e-4936-bf9e-8e006bd07ef0 None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:22.097655 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6542b8-3f58-48c5-85c6-a99f6abb77b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:22.098420 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6542b8-3f58-48c5-85c6-a99f6abb77b1 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:22.098834 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6542b8-3f58-48c5-85c6-a99f6abb77b1 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:22.099222 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6542b8-3f58-48c5-85c6-a99f6abb77b1 None admin] PATH_INFO: `/v3/roles` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:22.100179 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6542b8-3f58-48c5-85c6-a99f6abb77b1 None admin] RBAC: Adding query filter params (name=member) {{(pid=86827) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:34:22.100537 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6542b8-3f58-48c5-85c6-a99f6abb77b1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:22.102718 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6542b8-3f58-48c5-85c6-a99f6abb77b1 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:22.111946 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 128/249] 10.222.0.36 () {56 vars in 1101 bytes} [Wed Jul 27 10:34:21 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 10:34:22.118794 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6542b8-3f58-48c5-85c6-a99f6abb77b1 None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:22.125643 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1470fd-916b-4def-af17-2a7940fe8256 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:22.126469 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1470fd-916b-4def-af17-2a7940fe8256 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:22.126945 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1470fd-916b-4def-af17-2a7940fe8256 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:22.127366 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1470fd-916b-4def-af17-2a7940fe8256 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:22.128316 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1470fd-916b-4def-af17-2a7940fe8256 None admin] RBAC: Adding query filter params (role.id=80f67bfde4f5494e9c6d825aafb44e11, scope.project.id=046cff0c48ee4df282871e4eda9cf939, user.id=9a8d5697b327417aa265bb38fb008a75) {{(pid=86828) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:34:22.129305 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:22.130365 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1470fd-916b-4def-af17-2a7940fe8256 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:22.132492 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1470fd-916b-4def-af17-2a7940fe8256 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:22.141921 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 122/250] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Jul 27 10:34:21 2022] GET /identity/v3/role_assignments?user.id=9a8d5697b327417aa265bb38fb008a75&scope.project.id=046cff0c48ee4df282871e4eda9cf939&role.id=80f67bfde4f5494e9c6d825aafb44e11 => generated 259 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 10:34:22.151200 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ce0b6d-24af-4fbf-bba2-17271f46f2d6 None None] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:22.151816 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ce0b6d-24af-4fbf-bba2-17271f46f2d6 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:22.152293 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ce0b6d-24af-4fbf-bba2-17271f46f2d6 None None] PATH_INFO: `/` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:22.154131 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 123/251] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 10:34:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 10:34:22.160692 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bd15bf-9694-479d-9ef6-adc2c3a00394 None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:22.161291 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bd15bf-9694-479d-9ef6-adc2c3a00394 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:22.161765 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bd15bf-9694-479d-9ef6-adc2c3a00394 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:22.185474 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-21bd15bf-9694-479d-9ef6-adc2c3a00394 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:34:22.187149 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-21bd15bf-9694-479d-9ef6-adc2c3a00394 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:34:22.247178 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6542b8-3f58-48c5-85c6-a99f6abb77b1 None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:22.284480 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 124/252] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 10:34:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 10:34:22.297562 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741761de-1d97-4b78-ae0b-189aeb5ec955 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:22.298331 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741761de-1d97-4b78-ae0b-189aeb5ec955 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:22.298736 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741761de-1d97-4b78-ae0b-189aeb5ec955 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:22.299137 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741761de-1d97-4b78-ae0b-189aeb5ec955 None admin] PATH_INFO: `/v3/users/73fa0cd345004953a63f05de2ddc4ef4` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:22.300514 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741761de-1d97-4b78-ae0b-189aeb5ec955 None admin] RBAC: Authorizing `identity:get_user(user_id=73fa0cd345004953a63f05de2ddc4ef4)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:22.302117 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741761de-1d97-4b78-ae0b-189aeb5ec955 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:22.304318 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 129/253] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Jul 27 10:34:22 2022] GET /identity/v3/users/73fa0cd345004953a63f05de2ddc4ef4 => generated 297 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:34:22.310520 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bd15bf-9694-479d-9ef6-adc2c3a00394 None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:22.319052 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741761de-1d97-4b78-ae0b-189aeb5ec955 None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:22.405748 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:22.412394 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:22.426288 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bd15bf-9694-479d-9ef6-adc2c3a00394 None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:22.439730 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741761de-1d97-4b78-ae0b-189aeb5ec955 None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:22.464915 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc9aa28-763c-42e6-83ae-7ea4ae352bae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:22.465579 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc9aa28-763c-42e6-83ae-7ea4ae352bae None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:22.465865 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc9aa28-763c-42e6-83ae-7ea4ae352bae None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:22.466436 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc9aa28-763c-42e6-83ae-7ea4ae352bae None admin] PATH_INFO: `/v3/projects/046cff0c48ee4df282871e4eda9cf939` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:22.467463 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc9aa28-763c-42e6-83ae-7ea4ae352bae None admin] RBAC: Authorizing `identity:get_project(project_id=046cff0c48ee4df282871e4eda9cf939)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:22.469041 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc9aa28-763c-42e6-83ae-7ea4ae352bae None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:22.470881 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 125/254] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Jul 27 10:34:22 2022] GET /identity/v3/projects/046cff0c48ee4df282871e4eda9cf939 => generated 302 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 10:34:22.477256 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc9aa28-763c-42e6-83ae-7ea4ae352bae None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:22.478998 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bd056d-9e2b-4131-aa1c-37d63e45eba3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:22.479977 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bd056d-9e2b-4131-aa1c-37d63e45eba3 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:22.480473 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bd056d-9e2b-4131-aa1c-37d63e45eba3 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:22.481039 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bd056d-9e2b-4131-aa1c-37d63e45eba3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:22.493530 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42bd056d-9e2b-4131-aa1c-37d63e45eba3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 10:34:22.499632 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application [None req-42bd056d-9e2b-4131-aa1c-37d63e45eba3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 10:34:22.499632 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:34:22.499632 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:34:22.499632 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:34:22.499632 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:34:22.499632 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:34:22.499632 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:34:22.499632 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:34:22.499632 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:34:22.499632 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:34:22.499632 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:34:22.499632 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:34:22.499632 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 10:34:22.499632 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 10:34:22.499632 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 10:34:22.499632 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 10:34:22.500489 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 10:34:22.500489 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 10:34:22.500489 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 10:34:22.500489 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 10:34:22.500489 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:34:22.500489 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:34:22.500489 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:34:22.500489 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:34:22.500489 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:34:22.500489 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:34:22.500489 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:34:22.500489 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application with Lock( Jul 27 10:34:22.500489 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:34:22.500489 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:34:22.500489 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:34:22.500489 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:34:22.501257 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:34:22.501257 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:34:22.501257 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:34:22.501257 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:34:22.501257 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 10:34:22.501257 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 10:34:22.501257 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 10:34:22.501257 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 10:34:22.501257 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 10:34:22.501257 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 10:34:22.501257 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 10:34:22.501257 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Jul 27 10:34:22.503763 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 130/255] 10.222.0.36 () {56 vars in 1096 bytes} [Wed Jul 27 10:34:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 10:34:22.510316 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bd056d-9e2b-4131-aa1c-37d63e45eba3 None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:22.586282 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc9aa28-763c-42e6-83ae-7ea4ae352bae None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:22.623705 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bd056d-9e2b-4131-aa1c-37d63e45eba3 None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:22.624217 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa4e586-51d9-4e44-b323-0d00b87acb70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:22.624811 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa4e586-51d9-4e44-b323-0d00b87acb70 None admin] REQUEST_METHOD: `PUT` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:22.625094 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa4e586-51d9-4e44-b323-0d00b87acb70 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:22.625372 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa4e586-51d9-4e44-b323-0d00b87acb70 None admin] PATH_INFO: `/v3/projects/046cff0c48ee4df282871e4eda9cf939/users/73fa0cd345004953a63f05de2ddc4ef4/roles/4c9fc0b3baed441da2126be41cc6b04a` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:22.627631 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa4e586-51d9-4e44-b323-0d00b87acb70 None admin] RBAC: Authorizing `identity:create_grant(project_id=046cff0c48ee4df282871e4eda9cf939, user_id=73fa0cd345004953a63f05de2ddc4ef4, role_id=4c9fc0b3baed441da2126be41cc6b04a)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:22.629241 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa4e586-51d9-4e44-b323-0d00b87acb70 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:22.651149 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 126/256] 10.222.0.36 () {58 vars in 1439 bytes} [Wed Jul 27 10:34:22 2022] PUT /identity/v3/projects/046cff0c48ee4df282871e4eda9cf939/users/73fa0cd345004953a63f05de2ddc4ef4/roles/4c9fc0b3baed441da2126be41cc6b04a => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 10:34:22.652519 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85ae1dc-5f5d-4e7d-b970-0b462d755ee2 None None] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:22.652812 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85ae1dc-5f5d-4e7d-b970-0b462d755ee2 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:22.653195 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85ae1dc-5f5d-4e7d-b970-0b462d755ee2 None None] PATH_INFO: `/` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:22.654549 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 127/257] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 10:34:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 10:34:22.662445 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4cca64-a3ee-4808-9e03-f21abdd62922 None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:22.662740 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4cca64-a3ee-4808-9e03-f21abdd62922 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:22.662959 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4cca64-a3ee-4808-9e03-f21abdd62922 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:22.666896 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1e72c2-7154-41aa-a85c-375f0108020f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:22.667765 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1e72c2-7154-41aa-a85c-375f0108020f None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:22.668321 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1e72c2-7154-41aa-a85c-375f0108020f None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:22.668808 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1e72c2-7154-41aa-a85c-375f0108020f None admin] PATH_INFO: `/v3/roles` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:22.669750 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1e72c2-7154-41aa-a85c-375f0108020f None admin] RBAC: Adding query filter params (name=admin) {{(pid=86827) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:34:22.670290 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1e72c2-7154-41aa-a85c-375f0108020f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:22.672074 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1e72c2-7154-41aa-a85c-375f0108020f None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:22.675120 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:22.683578 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 131/258] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jul 27 10:34:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 10:34:22.690489 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1e72c2-7154-41aa-a85c-375f0108020f None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:22.703899 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-fb4cca64-a3ee-4808-9e03-f21abdd62922 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:34:22.705134 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-fb4cca64-a3ee-4808-9e03-f21abdd62922 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:34:22.725262 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:22.817269 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 128/259] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 10:34:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 27 10:34:22.858945 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4cca64-a3ee-4808-9e03-f21abdd62922 None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:22.929487 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1e72c2-7154-41aa-a85c-375f0108020f None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:22.982681 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f3341b-dba2-4057-8f1a-7bdce4e204c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:22.983913 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f3341b-dba2-4057-8f1a-7bdce4e204c0 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:22.984379 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f3341b-dba2-4057-8f1a-7bdce4e204c0 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:22.984832 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f3341b-dba2-4057-8f1a-7bdce4e204c0 None admin] PATH_INFO: `/v3/users/533e1f1f20424aeda4aea04123be56d6` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:22.992965 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f3341b-dba2-4057-8f1a-7bdce4e204c0 None admin] RBAC: Authorizing `identity:get_user(user_id=533e1f1f20424aeda4aea04123be56d6)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:22.994641 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f3341b-dba2-4057-8f1a-7bdce4e204c0 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:22.999912 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 132/260] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Jul 27 10:34:22 2022] GET /identity/v3/users/533e1f1f20424aeda4aea04123be56d6 => generated 283 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:34:23.028708 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f3341b-dba2-4057-8f1a-7bdce4e204c0 None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:23.098639 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4cca64-a3ee-4808-9e03-f21abdd62922 None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:23.131814 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:23.194048 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f3341b-dba2-4057-8f1a-7bdce4e204c0 None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:23.227312 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cf1f5c-5ccf-4917-8336-d7871d83cbb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:23.228087 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cf1f5c-5ccf-4917-8336-d7871d83cbb8 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:23.228369 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cf1f5c-5ccf-4917-8336-d7871d83cbb8 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:23.228640 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cf1f5c-5ccf-4917-8336-d7871d83cbb8 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:23.231418 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5816cf1-0100-46be-9ed5-5df1ee5b278d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:23.232528 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5816cf1-0100-46be-9ed5-5df1ee5b278d None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:23.232991 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5816cf1-0100-46be-9ed5-5df1ee5b278d None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:23.233445 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5816cf1-0100-46be-9ed5-5df1ee5b278d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:23.283225 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cf1f5c-5ccf-4917-8336-d7871d83cbb8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 10:34:23.288245 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application [None req-96cf1f5c-5ccf-4917-8336-d7871d83cbb8 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 27 10:34:23.288245 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:34:23.288245 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:34:23.288245 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:34:23.288245 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:34:23.288245 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:34:23.288245 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:34:23.288245 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:34:23.288245 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:34:23.288245 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:34:23.288245 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:34:23.288245 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:34:23.288245 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 10:34:23.288245 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 10:34:23.288245 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 10:34:23.288245 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 10:34:23.289243 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 10:34:23.289243 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 10:34:23.289243 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 10:34:23.289243 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 10:34:23.289243 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:34:23.289243 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:34:23.289243 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:34:23.289243 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:34:23.289243 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:34:23.289243 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:34:23.289243 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:34:23.289243 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application with Lock( Jul 27 10:34:23.289243 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:34:23.289243 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:34:23.289243 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:34:23.289243 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:34:23.290039 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:34:23.290039 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:34:23.290039 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:34:23.290039 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:34:23.290039 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 10:34:23.290039 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 10:34:23.290039 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 10:34:23.290039 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 10:34:23.290039 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 10:34:23.290039 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 10:34:23.290039 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 27 10:34:23.290039 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Jul 27 10:34:23.290039 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 129/261] 10.222.0.36 () {56 vars in 1114 bytes} [Wed Jul 27 10:34:22 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 431 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 10:34:23.299708 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cf1f5c-5ccf-4917-8336-d7871d83cbb8 None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:23.318057 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5816cf1-0100-46be-9ed5-5df1ee5b278d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:23.319842 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5816cf1-0100-46be-9ed5-5df1ee5b278d None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:23.321401 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:23.401215 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:23.446054 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 133/262] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jul 27 10:34:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 27 10:34:23.451984 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5816cf1-0100-46be-9ed5-5df1ee5b278d None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:23.500027 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cf1f5c-5ccf-4917-8336-d7871d83cbb8 None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:23.565397 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5816cf1-0100-46be-9ed5-5df1ee5b278d None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:23.567299 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defe3a0a-7157-454e-ae12-747bf0a9b4b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:23.568804 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defe3a0a-7157-454e-ae12-747bf0a9b4b2 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:23.568804 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defe3a0a-7157-454e-ae12-747bf0a9b4b2 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:23.568804 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defe3a0a-7157-454e-ae12-747bf0a9b4b2 None admin] PATH_INFO: `/v3/roles` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:23.569643 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defe3a0a-7157-454e-ae12-747bf0a9b4b2 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86828) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:34:23.570037 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defe3a0a-7157-454e-ae12-747bf0a9b4b2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:23.572173 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defe3a0a-7157-454e-ae12-747bf0a9b4b2 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:23.592646 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 130/263] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Jul 27 10:34:23 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:34:23.597847 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defe3a0a-7157-454e-ae12-747bf0a9b4b2 None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:23.607177 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578a79bb-4536-44f2-85aa-b4286ba63ba1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:23.608163 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578a79bb-4536-44f2-85aa-b4286ba63ba1 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:23.609469 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578a79bb-4536-44f2-85aa-b4286ba63ba1 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:23.609902 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578a79bb-4536-44f2-85aa-b4286ba63ba1 None admin] PATH_INFO: `/v3/projects/046cff0c48ee4df282871e4eda9cf939` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:23.615878 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578a79bb-4536-44f2-85aa-b4286ba63ba1 None admin] RBAC: Authorizing `identity:get_project(project_id=046cff0c48ee4df282871e4eda9cf939)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:23.618486 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578a79bb-4536-44f2-85aa-b4286ba63ba1 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:23.621407 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 134/264] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Jul 27 10:34:23 2022] GET /identity/v3/projects/046cff0c48ee4df282871e4eda9cf939 => generated 302 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:34:23.627216 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578a79bb-4536-44f2-85aa-b4286ba63ba1 None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:23.661250 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:23.713902 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:23.741248 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578a79bb-4536-44f2-85aa-b4286ba63ba1 None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:23.780097 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13109bb9-e919-46d9-bf21-d50a3c529f7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:23.781082 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13109bb9-e919-46d9-bf21-d50a3c529f7e None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:23.782160 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13109bb9-e919-46d9-bf21-d50a3c529f7e None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:23.782676 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13109bb9-e919-46d9-bf21-d50a3c529f7e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:23.783908 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13109bb9-e919-46d9-bf21-d50a3c529f7e None admin] RBAC: Adding query filter params (role.id=21156560eb6b436bb567a40b4976f3f4, scope.project.id=046cff0c48ee4df282871e4eda9cf939, user.id=533e1f1f20424aeda4aea04123be56d6) {{(pid=86827) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:34:23.784432 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13109bb9-e919-46d9-bf21-d50a3c529f7e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:23.786234 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13109bb9-e919-46d9-bf21-d50a3c529f7e None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:23.793129 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 135/265] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Jul 27 10:34:23 2022] GET /identity/v3/role_assignments?user.id=533e1f1f20424aeda4aea04123be56d6&scope.project.id=046cff0c48ee4df282871e4eda9cf939&role.id=21156560eb6b436bb567a40b4976f3f4 => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 10:34:23.827158 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defe3a0a-7157-454e-ae12-747bf0a9b4b2 None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:23.877339 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e102fe-c5a1-42fc-bff3-ab4604b34184 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:23.878383 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e102fe-c5a1-42fc-bff3-ab4604b34184 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:23.878922 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e102fe-c5a1-42fc-bff3-ab4604b34184 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:23.879405 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e102fe-c5a1-42fc-bff3-ab4604b34184 None admin] PATH_INFO: `/v3/users/533e1f1f20424aeda4aea04123be56d6` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:23.880953 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e102fe-c5a1-42fc-bff3-ab4604b34184 None admin] RBAC: Authorizing `identity:get_user(user_id=533e1f1f20424aeda4aea04123be56d6)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:23.882860 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e102fe-c5a1-42fc-bff3-ab4604b34184 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:23.885669 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 131/266] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Jul 27 10:34:23 2022] GET /identity/v3/users/533e1f1f20424aeda4aea04123be56d6 => generated 283 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:34:23.891196 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13109bb9-e919-46d9-bf21-d50a3c529f7e None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:24.023168 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:24.030844 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13109bb9-e919-46d9-bf21-d50a3c529f7e None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:24.076705 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e968e3-a48a-4ebd-b362-7d8b01a87a3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:24.077381 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e968e3-a48a-4ebd-b362-7d8b01a87a3c None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:24.077667 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e968e3-a48a-4ebd-b362-7d8b01a87a3c None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:24.077943 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e968e3-a48a-4ebd-b362-7d8b01a87a3c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:24.170042 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e968e3-a48a-4ebd-b362-7d8b01a87a3c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:24.171884 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e968e3-a48a-4ebd-b362-7d8b01a87a3c None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:24.295367 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:24.305996 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 136/267] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jul 27 10:34:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 10:34:24.311812 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e102fe-c5a1-42fc-bff3-ab4604b34184 None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:24.433591 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e102fe-c5a1-42fc-bff3-ab4604b34184 None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:24.474084 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac7f80c-f519-4d89-97f1-f5f5b5e14c60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:24.474773 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac7f80c-f519-4d89-97f1-f5f5b5e14c60 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:24.475062 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac7f80c-f519-4d89-97f1-f5f5b5e14c60 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:24.475342 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac7f80c-f519-4d89-97f1-f5f5b5e14c60 None admin] PATH_INFO: `/v3/projects/046cff0c48ee4df282871e4eda9cf939` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:24.477080 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac7f80c-f519-4d89-97f1-f5f5b5e14c60 None admin] RBAC: Authorizing `identity:get_project(project_id=046cff0c48ee4df282871e4eda9cf939)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:24.478561 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac7f80c-f519-4d89-97f1-f5f5b5e14c60 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:24.480586 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 132/268] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Jul 27 10:34:24 2022] GET /identity/v3/projects/046cff0c48ee4df282871e4eda9cf939 => generated 302 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:34:24.486224 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e968e3-a48a-4ebd-b362-7d8b01a87a3c None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:24.642004 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e968e3-a48a-4ebd-b362-7d8b01a87a3c None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:24.685493 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74eb1a2d-c64c-484e-9dc3-345aa86cc8c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:24.686473 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74eb1a2d-c64c-484e-9dc3-345aa86cc8c3 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:24.687027 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74eb1a2d-c64c-484e-9dc3-345aa86cc8c3 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:24.687509 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74eb1a2d-c64c-484e-9dc3-345aa86cc8c3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:24.688499 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74eb1a2d-c64c-484e-9dc3-345aa86cc8c3 None admin] RBAC: Adding query filter params (role.id=c3341f95596344309327efff99a20d5c, scope.project.id=046cff0c48ee4df282871e4eda9cf939, user.id=533e1f1f20424aeda4aea04123be56d6) {{(pid=86827) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:34:24.689004 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74eb1a2d-c64c-484e-9dc3-345aa86cc8c3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:24.690857 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74eb1a2d-c64c-484e-9dc3-345aa86cc8c3 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:24.698137 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 137/269] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Jul 27 10:34:24 2022] GET /identity/v3/role_assignments?user.id=533e1f1f20424aeda4aea04123be56d6&scope.project.id=046cff0c48ee4df282871e4eda9cf939&role.id=c3341f95596344309327efff99a20d5c => generated 608 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:34:24.858515 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3758aa3d-4f6d-41fe-94f5-853784b47093 None None] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:24.859070 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3758aa3d-4f6d-41fe-94f5-853784b47093 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:24.859070 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3758aa3d-4f6d-41fe-94f5-853784b47093 None None] PATH_INFO: `/` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:24.860583 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 133/270] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 10:34:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 10:34:24.866141 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277659a2-146d-4e8d-a76f-fb488a22d1a0 None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:24.866773 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277659a2-146d-4e8d-a76f-fb488a22d1a0 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:24.867280 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277659a2-146d-4e8d-a76f-fb488a22d1a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:24.890134 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-277659a2-146d-4e8d-a76f-fb488a22d1a0 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:34:24.891716 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-277659a2-146d-4e8d-a76f-fb488a22d1a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:34:24.964057 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:24.976344 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 138/271] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 10:34:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 10:34:24.999590 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3758aa3d-4f6d-41fe-94f5-853784b47093 None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:25.004975 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:25.085398 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650111c0-7897-4823-8aee-572650d5e292 None None] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:25.085782 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650111c0-7897-4823-8aee-572650d5e292 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:25.086068 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650111c0-7897-4823-8aee-572650d5e292 None None] PATH_INFO: `/` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:25.087739 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 139/272] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 10:34:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 10:34:25.095890 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84f9f27-d9e4-4e61-8d38-599a86ac86f8 None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:25.096314 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84f9f27-d9e4-4e61-8d38-599a86ac86f8 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:25.096684 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84f9f27-d9e4-4e61-8d38-599a86ac86f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:25.118029 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-f84f9f27-d9e4-4e61-8d38-599a86ac86f8 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:34:25.119287 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3758aa3d-4f6d-41fe-94f5-853784b47093 None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:25.119444 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-f84f9f27-d9e4-4e61-8d38-599a86ac86f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:34:25.156560 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276c6579-3ad7-4545-bc66-08f504b8dff3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:25.157479 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276c6579-3ad7-4545-bc66-08f504b8dff3 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:25.157935 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276c6579-3ad7-4545-bc66-08f504b8dff3 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:25.158219 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276c6579-3ad7-4545-bc66-08f504b8dff3 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:25.169067 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276c6579-3ad7-4545-bc66-08f504b8dff3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 10:34:25.174572 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application [None req-276c6579-3ad7-4545-bc66-08f504b8dff3 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 27 10:34:25.174572 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:34:25.174572 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:34:25.174572 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:34:25.174572 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:34:25.174572 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:34:25.174572 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:34:25.174572 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:34:25.174572 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:34:25.174572 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:34:25.174572 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:34:25.174572 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:34:25.174572 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 10:34:25.174572 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 10:34:25.174572 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 10:34:25.174572 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 10:34:25.175679 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 10:34:25.175679 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 10:34:25.175679 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 10:34:25.175679 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 10:34:25.175679 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:34:25.175679 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:34:25.175679 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:34:25.175679 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:34:25.175679 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:34:25.175679 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:34:25.175679 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:34:25.175679 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application with Lock( Jul 27 10:34:25.175679 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:34:25.175679 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:34:25.175679 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:34:25.175679 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:34:25.176478 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:34:25.176478 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:34:25.176478 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:34:25.176478 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:34:25.176478 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 10:34:25.176478 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 10:34:25.176478 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 10:34:25.176478 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 10:34:25.176478 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 10:34:25.176478 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 10:34:25.176478 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 27 10:34:25.176478 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Jul 27 10:34:25.177066 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 134/273] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jul 27 10:34:24 2022] GET /identity/v3/roles/reader => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 10:34:25.183206 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276c6579-3ad7-4545-bc66-08f504b8dff3 None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:25.211972 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 140/274] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 10:34:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 27 10:34:25.213705 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e012bd-c3d5-4cf5-9a1c-b6cafbcfcce6 None None] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:25.214064 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e012bd-c3d5-4cf5-9a1c-b6cafbcfcce6 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:25.214409 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e012bd-c3d5-4cf5-9a1c-b6cafbcfcce6 None None] PATH_INFO: `/` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:25.215932 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 141/275] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 10:34:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 10:34:25.220898 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcf0ec9-16f0-4657-a452-98bba8f28a95 None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:25.221257 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcf0ec9-16f0-4657-a452-98bba8f28a95 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:25.221605 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcf0ec9-16f0-4657-a452-98bba8f28a95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:25.240699 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-7bcf0ec9-16f0-4657-a452-98bba8f28a95 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:34:25.241988 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-7bcf0ec9-16f0-4657-a452-98bba8f28a95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:34:25.261365 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:25.300570 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276c6579-3ad7-4545-bc66-08f504b8dff3 None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:25.315094 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 142/276] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 10:34:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 27 10:34:25.322822 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcf0ec9-16f0-4657-a452-98bba8f28a95 None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:25.341038 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a151e7c-8d39-4541-8df1-dbd2cf251590 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:25.341669 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a151e7c-8d39-4541-8df1-dbd2cf251590 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:25.341946 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a151e7c-8d39-4541-8df1-dbd2cf251590 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:25.342218 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a151e7c-8d39-4541-8df1-dbd2cf251590 None admin] PATH_INFO: `/v3/roles` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:25.343010 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a151e7c-8d39-4541-8df1-dbd2cf251590 None admin] RBAC: Adding query filter params (name=reader) {{(pid=86828) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:34:25.343315 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a151e7c-8d39-4541-8df1-dbd2cf251590 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:25.344747 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a151e7c-8d39-4541-8df1-dbd2cf251590 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:25.359006 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 135/277] 10.222.0.36 () {56 vars in 1101 bytes} [Wed Jul 27 10:34:25 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:34:25.359006 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a151e7c-8d39-4541-8df1-dbd2cf251590 None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:25.382425 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:25.447472 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcf0ec9-16f0-4657-a452-98bba8f28a95 None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:25.472118 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a151e7c-8d39-4541-8df1-dbd2cf251590 None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:25.484800 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b6b9f8-7d6b-4522-b412-a8a7540fe794 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:25.485455 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b6b9f8-7d6b-4522-b412-a8a7540fe794 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:25.485739 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b6b9f8-7d6b-4522-b412-a8a7540fe794 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:25.486012 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b6b9f8-7d6b-4522-b412-a8a7540fe794 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:25.487178 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b6b9f8-7d6b-4522-b412-a8a7540fe794 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:25.488700 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b6b9f8-7d6b-4522-b412-a8a7540fe794 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:25.490471 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 143/278] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 10:34:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:34:25.491172 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b6b9f8-7d6b-4522-b412-a8a7540fe794 None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:25.507624 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:25.518008 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e990d64-94ff-459a-860f-7f42a10441cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:25.518685 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e990d64-94ff-459a-860f-7f42a10441cf None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:25.519020 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e990d64-94ff-459a-860f-7f42a10441cf None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:25.519299 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e990d64-94ff-459a-860f-7f42a10441cf None admin] PATH_INFO: `/v3/roles/member` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:25.529664 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e990d64-94ff-459a-860f-7f42a10441cf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 10:34:25.535431 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application [None req-0e990d64-94ff-459a-860f-7f42a10441cf None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 27 10:34:25.535431 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:34:25.535431 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:34:25.535431 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:34:25.535431 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:34:25.535431 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:34:25.535431 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:34:25.535431 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:34:25.535431 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:34:25.535431 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:34:25.535431 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:34:25.535431 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:34:25.535431 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 10:34:25.535431 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 10:34:25.535431 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 10:34:25.535431 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 10:34:25.536543 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 10:34:25.536543 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 10:34:25.536543 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 10:34:25.536543 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 10:34:25.536543 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:34:25.536543 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:34:25.536543 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:34:25.536543 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:34:25.536543 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:34:25.536543 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:34:25.536543 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:34:25.536543 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application with Lock( Jul 27 10:34:25.536543 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:34:25.536543 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:34:25.536543 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:34:25.536543 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:34:25.537527 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:34:25.537527 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:34:25.537527 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:34:25.537527 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:34:25.537527 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 10:34:25.537527 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 10:34:25.537527 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 10:34:25.537527 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 10:34:25.537527 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 10:34:25.537527 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 10:34:25.537527 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 27 10:34:25.537527 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Jul 27 10:34:25.537527 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 136/279] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jul 27 10:34:25 2022] GET /identity/v3/roles/member => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 10:34:25.537527 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e990d64-94ff-459a-860f-7f42a10441cf None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:25.605709 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b6b9f8-7d6b-4522-b412-a8a7540fe794 None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:25.624129 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:25.643368 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f190e30e-7165-4104-97cb-1d995977d0dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:25.643977 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f190e30e-7165-4104-97cb-1d995977d0dc None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:25.644215 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f190e30e-7165-4104-97cb-1d995977d0dc None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:25.644488 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f190e30e-7165-4104-97cb-1d995977d0dc None admin] PATH_INFO: `/v3/users/9a8d5697b327417aa265bb38fb008a75` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:25.645651 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f190e30e-7165-4104-97cb-1d995977d0dc None admin] RBAC: Authorizing `identity:get_user(user_id=9a8d5697b327417aa265bb38fb008a75)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:25.646875 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e990d64-94ff-459a-860f-7f42a10441cf None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:25.647116 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f190e30e-7165-4104-97cb-1d995977d0dc None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:25.648959 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 144/280] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Jul 27 10:34:25 2022] GET /identity/v3/users/9a8d5697b327417aa265bb38fb008a75 => generated 297 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:34:25.649496 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f190e30e-7165-4104-97cb-1d995977d0dc None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:25.687968 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24972d5-9483-45f2-88ed-46c38bd76573 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:25.688594 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24972d5-9483-45f2-88ed-46c38bd76573 None admin] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:25.688878 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24972d5-9483-45f2-88ed-46c38bd76573 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:25.689152 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24972d5-9483-45f2-88ed-46c38bd76573 None admin] PATH_INFO: `/v3/users` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:25.689838 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24972d5-9483-45f2-88ed-46c38bd76573 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:25.691318 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24972d5-9483-45f2-88ed-46c38bd76573 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:25.719883 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 137/281] 10.222.0.36 () {60 vars in 1132 bytes} [Wed Jul 27 10:34:25 2022] POST /identity/v3/users => generated 293 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 10:34:25.719883 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24972d5-9483-45f2-88ed-46c38bd76573 None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:25.769842 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:25.777952 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f190e30e-7165-4104-97cb-1d995977d0dc None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:25.817327 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b0e9d4-9ac8-4fa5-8dc9-5fb8a680b28c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:25.817967 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b0e9d4-9ac8-4fa5-8dc9-5fb8a680b28c None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:25.818248 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b0e9d4-9ac8-4fa5-8dc9-5fb8a680b28c None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:25.818531 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b0e9d4-9ac8-4fa5-8dc9-5fb8a680b28c None admin] PATH_INFO: `/v3/roles` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:25.819298 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b0e9d4-9ac8-4fa5-8dc9-5fb8a680b28c None admin] RBAC: Adding query filter params (name=member) {{(pid=86827) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:34:25.819605 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b0e9d4-9ac8-4fa5-8dc9-5fb8a680b28c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:25.821168 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b0e9d4-9ac8-4fa5-8dc9-5fb8a680b28c None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:25.831306 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 145/282] 10.222.0.36 () {56 vars in 1101 bytes} [Wed Jul 27 10:34:25 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:34:25.832444 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58c47df-60b9-4e23-85f6-ab95b0a31b5f None None] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:25.832727 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58c47df-60b9-4e23-85f6-ab95b0a31b5f None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:25.833008 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58c47df-60b9-4e23-85f6-ab95b0a31b5f None None] PATH_INFO: `/` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:25.834520 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 146/283] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 10:34:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 10:34:25.837268 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58c47df-60b9-4e23-85f6-ab95b0a31b5f None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:25.843461 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24972d5-9483-45f2-88ed-46c38bd76573 None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:25.883061 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1ac54e-7c21-453a-97e7-5c23d334ccfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:25.883776 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1ac54e-7c21-453a-97e7-5c23d334ccfb None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:25.884166 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1ac54e-7c21-453a-97e7-5c23d334ccfb None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:25.884768 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1ac54e-7c21-453a-97e7-5c23d334ccfb None admin] PATH_INFO: `/v3/projects/046cff0c48ee4df282871e4eda9cf939` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:25.887341 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1ac54e-7c21-453a-97e7-5c23d334ccfb None admin] RBAC: Authorizing `identity:get_project(project_id=046cff0c48ee4df282871e4eda9cf939)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:25.889311 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1ac54e-7c21-453a-97e7-5c23d334ccfb None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:25.891624 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 138/284] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Jul 27 10:34:25 2022] GET /identity/v3/projects/046cff0c48ee4df282871e4eda9cf939 => generated 302 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:34:25.892927 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367d0fbf-d716-4482-875d-b5dbb67de3f0 None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:25.893198 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367d0fbf-d716-4482-875d-b5dbb67de3f0 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:25.893481 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367d0fbf-d716-4482-875d-b5dbb67de3f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:25.903303 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:25.913767 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-367d0fbf-d716-4482-875d-b5dbb67de3f0 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:34:25.915067 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-367d0fbf-d716-4482-875d-b5dbb67de3f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:34:25.970962 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58c47df-60b9-4e23-85f6-ab95b0a31b5f None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:26.001930 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 139/285] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 10:34:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 10:34:26.002637 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367d0fbf-d716-4482-875d-b5dbb67de3f0 None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:26.014632 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae9dc20-f7ee-4ebb-90b0-68a2657c1c4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:26.015551 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae9dc20-f7ee-4ebb-90b0-68a2657c1c4d None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:26.015551 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae9dc20-f7ee-4ebb-90b0-68a2657c1c4d None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:26.015709 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae9dc20-f7ee-4ebb-90b0-68a2657c1c4d None admin] PATH_INFO: `/v3/users/1224eb09d9ed4d728fcd0460454635b2` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:26.034366 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae9dc20-f7ee-4ebb-90b0-68a2657c1c4d None admin] RBAC: Authorizing `identity:get_user(user_id=1224eb09d9ed4d728fcd0460454635b2)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:26.035961 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae9dc20-f7ee-4ebb-90b0-68a2657c1c4d None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:26.037683 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 147/286] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Jul 27 10:34:25 2022] GET /identity/v3/users/1224eb09d9ed4d728fcd0460454635b2 => generated 293 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 10:34:26.038895 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae9dc20-f7ee-4ebb-90b0-68a2657c1c4d None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:26.057680 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:26.126762 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367d0fbf-d716-4482-875d-b5dbb67de3f0 None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:26.154462 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae9dc20-f7ee-4ebb-90b0-68a2657c1c4d None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:26.168705 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:26.173195 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f26164a-343c-48f2-af98-c782c9b8829e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:26.173850 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f26164a-343c-48f2-af98-c782c9b8829e None admin] REQUEST_METHOD: `PUT` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:26.174129 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f26164a-343c-48f2-af98-c782c9b8829e None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:26.174403 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f26164a-343c-48f2-af98-c782c9b8829e None admin] PATH_INFO: `/v3/projects/046cff0c48ee4df282871e4eda9cf939/users/9a8d5697b327417aa265bb38fb008a75/roles/80f67bfde4f5494e9c6d825aafb44e11` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:26.176735 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f26164a-343c-48f2-af98-c782c9b8829e None admin] RBAC: Authorizing `identity:create_grant(project_id=046cff0c48ee4df282871e4eda9cf939, user_id=9a8d5697b327417aa265bb38fb008a75, role_id=80f67bfde4f5494e9c6d825aafb44e11)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:26.178507 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f26164a-343c-48f2-af98-c782c9b8829e None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:26.196474 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 140/287] 10.222.0.36 () {58 vars in 1439 bytes} [Wed Jul 27 10:34:26 2022] PUT /identity/v3/projects/046cff0c48ee4df282871e4eda9cf939/users/9a8d5697b327417aa265bb38fb008a75/roles/80f67bfde4f5494e9c6d825aafb44e11 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:34:26.196786 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d65689f-d0e3-44fc-b3aa-055ab5192640 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:26.197282 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f26164a-343c-48f2-af98-c782c9b8829e None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:26.198029 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d65689f-d0e3-44fc-b3aa-055ab5192640 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:26.198502 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d65689f-d0e3-44fc-b3aa-055ab5192640 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:26.198989 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d65689f-d0e3-44fc-b3aa-055ab5192640 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:26.213234 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d65689f-d0e3-44fc-b3aa-055ab5192640 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 10:34:26.219065 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application [None req-1d65689f-d0e3-44fc-b3aa-055ab5192640 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 27 10:34:26.219065 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:34:26.219065 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:34:26.219065 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:34:26.219065 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:34:26.219065 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:34:26.219065 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:34:26.219065 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:34:26.219065 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:34:26.219065 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:34:26.219065 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:34:26.219065 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:34:26.219065 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 10:34:26.219065 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 10:34:26.219065 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 10:34:26.219065 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 10:34:26.219990 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 10:34:26.219990 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 10:34:26.219990 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 10:34:26.219990 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 10:34:26.219990 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:34:26.219990 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:34:26.219990 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:34:26.219990 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:34:26.219990 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:34:26.219990 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:34:26.219990 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:34:26.219990 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application with Lock( Jul 27 10:34:26.219990 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:34:26.219990 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:34:26.219990 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:34:26.219990 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:34:26.220772 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:34:26.220772 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:34:26.220772 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:34:26.220772 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:34:26.220772 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 10:34:26.220772 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 10:34:26.220772 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 10:34:26.220772 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 10:34:26.220772 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 10:34:26.220772 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 10:34:26.220772 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 27 10:34:26.220772 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Jul 27 10:34:26.223230 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 148/288] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jul 27 10:34:26 2022] GET /identity/v3/roles/member => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 10:34:26.229221 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d65689f-d0e3-44fc-b3aa-055ab5192640 None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:26.386487 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:26.390107 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f26164a-343c-48f2-af98-c782c9b8829e None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:26.418709 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d65689f-d0e3-44fc-b3aa-055ab5192640 None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:26.436386 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdff947c-77a7-4e54-ab67-fbaf5725266b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:26.437500 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdff947c-77a7-4e54-ab67-fbaf5725266b None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:26.437978 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdff947c-77a7-4e54-ab67-fbaf5725266b None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:26.438727 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdff947c-77a7-4e54-ab67-fbaf5725266b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:26.439753 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdff947c-77a7-4e54-ab67-fbaf5725266b None admin] RBAC: Adding query filter params (role.id=4c9fc0b3baed441da2126be41cc6b04a, user.id=1224eb09d9ed4d728fcd0460454635b2, scope.system=all) {{(pid=86828) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:34:26.440274 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdff947c-77a7-4e54-ab67-fbaf5725266b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:26.442347 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdff947c-77a7-4e54-ab67-fbaf5725266b None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:26.450662 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 141/289] 10.222.0.36 () {56 vars in 1308 bytes} [Wed Jul 27 10:34:26 2022] GET /identity/v3/role_assignments?user.id=1224eb09d9ed4d728fcd0460454635b2&scope.system=all&role.id=4c9fc0b3baed441da2126be41cc6b04a => generated 226 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:34:26.459955 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a8df3b-b861-4598-8564-dc8316fdd54f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:26.460602 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a8df3b-b861-4598-8564-dc8316fdd54f None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:26.461500 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a8df3b-b861-4598-8564-dc8316fdd54f None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:26.461781 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a8df3b-b861-4598-8564-dc8316fdd54f None admin] PATH_INFO: `/v3/roles` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:26.462526 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a8df3b-b861-4598-8564-dc8316fdd54f None admin] RBAC: Adding query filter params (name=member) {{(pid=86827) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:34:26.462870 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a8df3b-b861-4598-8564-dc8316fdd54f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:26.464301 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a8df3b-b861-4598-8564-dc8316fdd54f None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:26.474760 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 149/290] 10.222.0.36 () {56 vars in 1101 bytes} [Wed Jul 27 10:34:26 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 10:34:26.481640 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdff947c-77a7-4e54-ab67-fbaf5725266b None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:26.615507 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdff947c-77a7-4e54-ab67-fbaf5725266b None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:26.662420 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835b9d1b-b112-431c-81dd-0da773b94f57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:26.663456 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835b9d1b-b112-431c-81dd-0da773b94f57 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:26.664498 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835b9d1b-b112-431c-81dd-0da773b94f57 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:26.664972 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835b9d1b-b112-431c-81dd-0da773b94f57 None admin] PATH_INFO: `/v3/users/73fa0cd345004953a63f05de2ddc4ef4` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:26.667786 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835b9d1b-b112-431c-81dd-0da773b94f57 None admin] RBAC: Authorizing `identity:get_user(user_id=73fa0cd345004953a63f05de2ddc4ef4)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:26.669629 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835b9d1b-b112-431c-81dd-0da773b94f57 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:26.672402 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 142/291] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Jul 27 10:34:26 2022] GET /identity/v3/users/73fa0cd345004953a63f05de2ddc4ef4 => generated 297 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:34:26.677171 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a8df3b-b861-4598-8564-dc8316fdd54f None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:26.728707 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:26.830316 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a8df3b-b861-4598-8564-dc8316fdd54f None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:26.877586 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a418aa14-f605-47c6-8ff4-15411b4bd9c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:26.878275 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a418aa14-f605-47c6-8ff4-15411b4bd9c9 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:26.878565 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a418aa14-f605-47c6-8ff4-15411b4bd9c9 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:26.878869 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a418aa14-f605-47c6-8ff4-15411b4bd9c9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:26.978328 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a418aa14-f605-47c6-8ff4-15411b4bd9c9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:26.979822 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a418aa14-f605-47c6-8ff4-15411b4bd9c9 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:27.017732 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:27.120054 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 150/292] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jul 27 10:34:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 443 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 10:34:27.129555 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835b9d1b-b112-431c-81dd-0da773b94f57 None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:27.183684 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:27.292420 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835b9d1b-b112-431c-81dd-0da773b94f57 None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:27.336818 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba27c385-1c25-40a6-8244-c142cc82bcfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:27.337472 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba27c385-1c25-40a6-8244-c142cc82bcfb None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:27.337757 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba27c385-1c25-40a6-8244-c142cc82bcfb None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:27.338036 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba27c385-1c25-40a6-8244-c142cc82bcfb None admin] PATH_INFO: `/v3/projects/046cff0c48ee4df282871e4eda9cf939` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:27.339243 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba27c385-1c25-40a6-8244-c142cc82bcfb None admin] RBAC: Authorizing `identity:get_project(project_id=046cff0c48ee4df282871e4eda9cf939)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:27.340707 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba27c385-1c25-40a6-8244-c142cc82bcfb None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:27.342718 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 143/293] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Jul 27 10:34:27 2022] GET /identity/v3/projects/046cff0c48ee4df282871e4eda9cf939 => generated 302 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:34:27.348221 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a418aa14-f605-47c6-8ff4-15411b4bd9c9 None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:27.498239 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a418aa14-f605-47c6-8ff4-15411b4bd9c9 None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:27.537853 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:27.550814 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff31235e-a5d8-4db9-86cd-c42849748049 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:27.551443 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff31235e-a5d8-4db9-86cd-c42849748049 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:27.551726 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff31235e-a5d8-4db9-86cd-c42849748049 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:27.552002 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff31235e-a5d8-4db9-86cd-c42849748049 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:27.552805 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff31235e-a5d8-4db9-86cd-c42849748049 None admin] RBAC: Adding query filter params (role.id=4c9fc0b3baed441da2126be41cc6b04a, scope.project.id=046cff0c48ee4df282871e4eda9cf939, user.id=73fa0cd345004953a63f05de2ddc4ef4) {{(pid=86827) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:34:27.553117 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff31235e-a5d8-4db9-86cd-c42849748049 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:27.555112 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff31235e-a5d8-4db9-86cd-c42849748049 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:27.561719 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 151/294] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Jul 27 10:34:27 2022] GET /identity/v3/role_assignments?user.id=73fa0cd345004953a63f05de2ddc4ef4&scope.project.id=046cff0c48ee4df282871e4eda9cf939&role.id=4c9fc0b3baed441da2126be41cc6b04a => generated 608 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:34:28.075454 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241c7b2a-412c-4e74-8b0e-e471422c3125 None None] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:28.075862 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241c7b2a-412c-4e74-8b0e-e471422c3125 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:28.076045 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241c7b2a-412c-4e74-8b0e-e471422c3125 None None] PATH_INFO: `/` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:28.077373 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 144/295] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 10:34:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 10:34:28.082786 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf69b21-fe01-4e3e-807f-78e992d9befa None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:28.083213 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf69b21-fe01-4e3e-807f-78e992d9befa None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:28.083381 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf69b21-fe01-4e3e-807f-78e992d9befa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:28.104836 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-daf69b21-fe01-4e3e-807f-78e992d9befa None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:34:28.106060 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-daf69b21-fe01-4e3e-807f-78e992d9befa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:34:28.187976 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 152/296] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 10:34:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 10:34:28.211874 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241c7b2a-412c-4e74-8b0e-e471422c3125 None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:28.307784 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa5e760-41d3-4a6e-a340-e16b9e66802d None None] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:28.308135 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa5e760-41d3-4a6e-a340-e16b9e66802d None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:28.308372 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa5e760-41d3-4a6e-a340-e16b9e66802d None None] PATH_INFO: `/` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:28.309975 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 153/297] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 10:34:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 10:34:28.314957 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa300565-92ed-41cd-8717-0cbff726d371 None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:28.315247 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa300565-92ed-41cd-8717-0cbff726d371 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:28.315529 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa300565-92ed-41cd-8717-0cbff726d371 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:28.326908 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:28.331368 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241c7b2a-412c-4e74-8b0e-e471422c3125 None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:28.337141 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-fa300565-92ed-41cd-8717-0cbff726d371 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:34:28.338332 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-fa300565-92ed-41cd-8717-0cbff726d371 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:34:28.374561 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eef3305-463b-477f-897a-fbfb5fc60637 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:28.375250 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eef3305-463b-477f-897a-fbfb5fc60637 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:28.375535 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eef3305-463b-477f-897a-fbfb5fc60637 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:28.375813 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eef3305-463b-477f-897a-fbfb5fc60637 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:28.387851 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eef3305-463b-477f-897a-fbfb5fc60637 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 10:34:28.393242 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application [None req-0eef3305-463b-477f-897a-fbfb5fc60637 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 27 10:34:28.393242 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:34:28.393242 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:34:28.393242 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:34:28.393242 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:34:28.393242 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:34:28.393242 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:34:28.393242 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:34:28.393242 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:34:28.393242 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:34:28.393242 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:34:28.393242 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:34:28.393242 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 10:34:28.393242 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 10:34:28.393242 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 10:34:28.393242 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 10:34:28.394089 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 10:34:28.394089 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 10:34:28.394089 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 10:34:28.394089 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 10:34:28.394089 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:34:28.394089 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:34:28.394089 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:34:28.394089 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:34:28.394089 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:34:28.394089 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:34:28.394089 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:34:28.394089 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application with Lock( Jul 27 10:34:28.394089 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:34:28.394089 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:34:28.394089 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:34:28.394089 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:34:28.394895 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:34:28.394895 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:34:28.394895 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:34:28.394895 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:34:28.394895 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 10:34:28.394895 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 10:34:28.394895 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 10:34:28.394895 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 10:34:28.394895 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 10:34:28.394895 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 10:34:28.394895 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 27 10:34:28.394895 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Jul 27 10:34:28.394895 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 145/298] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jul 27 10:34:28 2022] GET /identity/v3/roles/reader => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 10:34:28.401055 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eef3305-463b-477f-897a-fbfb5fc60637 None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:28.424368 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 154/299] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 10:34:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 27 10:34:28.446789 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa300565-92ed-41cd-8717-0cbff726d371 None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:28.511495 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eef3305-463b-477f-897a-fbfb5fc60637 None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:28.521652 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:28.551664 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a6fbc9-c022-4a1a-a6d9-6450afbed3c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:28.552500 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a6fbc9-c022-4a1a-a6d9-6450afbed3c7 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:28.552684 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a6fbc9-c022-4a1a-a6d9-6450afbed3c7 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:28.552931 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a6fbc9-c022-4a1a-a6d9-6450afbed3c7 None admin] PATH_INFO: `/v3/roles` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:28.553657 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a6fbc9-c022-4a1a-a6d9-6450afbed3c7 None admin] RBAC: Adding query filter params (name=reader) {{(pid=86828) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:34:28.553959 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a6fbc9-c022-4a1a-a6d9-6450afbed3c7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:28.555612 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a6fbc9-c022-4a1a-a6d9-6450afbed3c7 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:28.565276 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 146/300] 10.222.0.36 () {56 vars in 1101 bytes} [Wed Jul 27 10:34:28 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 10:34:28.576205 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a6fbc9-c022-4a1a-a6d9-6450afbed3c7 None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:28.583003 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa300565-92ed-41cd-8717-0cbff726d371 None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:28.611039 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:28.629970 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b180c50f-898d-4638-9d3f-1faae80219ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:28.630994 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b180c50f-898d-4638-9d3f-1faae80219ed None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:28.631506 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b180c50f-898d-4638-9d3f-1faae80219ed None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:28.632515 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b180c50f-898d-4638-9d3f-1faae80219ed None admin] PATH_INFO: `/v3/domains/default` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:28.634742 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b180c50f-898d-4638-9d3f-1faae80219ed None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:28.637297 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b180c50f-898d-4638-9d3f-1faae80219ed None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:28.644727 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 155/301] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 10:34:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 10:34:28.645268 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b180c50f-898d-4638-9d3f-1faae80219ed None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:28.714934 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a6fbc9-c022-4a1a-a6d9-6450afbed3c7 None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:28.749724 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8939c4-87ab-4b47-9433-930fe487a558 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:28.750644 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8939c4-87ab-4b47-9433-930fe487a558 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:28.750934 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8939c4-87ab-4b47-9433-930fe487a558 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:28.751214 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8939c4-87ab-4b47-9433-930fe487a558 None admin] PATH_INFO: `/v3/users/b740b641ba2545caa817805367676e8f` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:28.765874 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8939c4-87ab-4b47-9433-930fe487a558 None admin] RBAC: Authorizing `identity:get_user(user_id=b740b641ba2545caa817805367676e8f)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:28.767303 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8939c4-87ab-4b47-9433-930fe487a558 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:28.768130 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b180c50f-898d-4638-9d3f-1faae80219ed None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:28.774224 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 147/302] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Jul 27 10:34:28 2022] GET /identity/v3/users/b740b641ba2545caa817805367676e8f => generated 293 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:34:28.774224 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9959fe64-471b-4c98-a48a-7173fa5a35a2 None None] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:28.774224 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9959fe64-471b-4c98-a48a-7173fa5a35a2 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:28.774224 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9959fe64-471b-4c98-a48a-7173fa5a35a2 None None] PATH_INFO: `/` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:28.774224 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 148/303] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 10:34:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 10:34:28.776580 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9959fe64-471b-4c98-a48a-7173fa5a35a2 None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:28.796036 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:28.813431 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3344f6-1502-4c56-b5e4-d855447cdf65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:28.814956 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3344f6-1502-4c56-b5e4-d855447cdf65 None admin] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:28.815247 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3344f6-1502-4c56-b5e4-d855447cdf65 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:28.815529 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3344f6-1502-4c56-b5e4-d855447cdf65 None admin] PATH_INFO: `/v3/groups` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:28.816206 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3344f6-1502-4c56-b5e4-d855447cdf65 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:28.817744 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3344f6-1502-4c56-b5e4-d855447cdf65 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:28.838171 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 156/304] 10.222.0.36 () {60 vars in 1134 bytes} [Wed Jul 27 10:34:28 2022] POST /identity/v3/groups => generated 229 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 10:34:28.839592 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b1099c-3e74-4135-945f-1465e1ee7720 None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:28.839889 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b1099c-3e74-4135-945f-1465e1ee7720 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:28.840171 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b1099c-3e74-4135-945f-1465e1ee7720 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:28.867040 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-a7b1099c-3e74-4135-945f-1465e1ee7720 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:34:28.868421 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-a7b1099c-3e74-4135-945f-1465e1ee7720 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:34:28.872292 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:28.905805 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9959fe64-471b-4c98-a48a-7173fa5a35a2 None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:28.953583 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a54b35-3ddf-4758-a01e-99ddbeb35138 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:28.954229 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a54b35-3ddf-4758-a01e-99ddbeb35138 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:28.954514 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a54b35-3ddf-4758-a01e-99ddbeb35138 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:28.954888 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a54b35-3ddf-4758-a01e-99ddbeb35138 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:28.955712 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a54b35-3ddf-4758-a01e-99ddbeb35138 None admin] RBAC: Adding query filter params (role.id=80f67bfde4f5494e9c6d825aafb44e11, user.id=b740b641ba2545caa817805367676e8f, scope.system=all) {{(pid=86828) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:34:28.956082 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a54b35-3ddf-4758-a01e-99ddbeb35138 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:28.957801 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a54b35-3ddf-4758-a01e-99ddbeb35138 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:28.965289 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 149/305] 10.222.0.36 () {56 vars in 1308 bytes} [Wed Jul 27 10:34:28 2022] GET /identity/v3/role_assignments?user.id=b740b641ba2545caa817805367676e8f&scope.system=all&role.id=80f67bfde4f5494e9c6d825aafb44e11 => generated 226 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:34:28.980679 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 157/306] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 10:34:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 10:34:29.001196 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a54b35-3ddf-4758-a01e-99ddbeb35138 None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:29.067935 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821ceed2-8896-422b-8a93-50e127c53f7a None None] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:29.068263 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821ceed2-8896-422b-8a93-50e127c53f7a None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:29.068503 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821ceed2-8896-422b-8a93-50e127c53f7a None None] PATH_INFO: `/` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:29.069899 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 158/307] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 10:34:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 10:34:29.076452 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b136f08f-c870-4963-a131-a825654cc449 None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:29.076999 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b136f08f-c870-4963-a131-a825654cc449 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:29.077539 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b136f08f-c870-4963-a131-a825654cc449 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:29.102819 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-b136f08f-c870-4963-a131-a825654cc449 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:34:29.104886 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-b136f08f-c870-4963-a131-a825654cc449 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:34:29.124082 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a54b35-3ddf-4758-a01e-99ddbeb35138 None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:29.160903 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3043b0-b16d-427e-a485-b6381b43a701 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:29.161560 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3043b0-b16d-427e-a485-b6381b43a701 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:29.161852 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3043b0-b16d-427e-a485-b6381b43a701 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:29.162125 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3043b0-b16d-427e-a485-b6381b43a701 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:29.179544 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3043b0-b16d-427e-a485-b6381b43a701 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 10:34:29.186686 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application [None req-aa3043b0-b16d-427e-a485-b6381b43a701 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 27 10:34:29.186686 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:34:29.186686 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:34:29.186686 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:34:29.186686 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:34:29.186686 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:34:29.186686 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:34:29.186686 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:34:29.186686 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:34:29.186686 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:34:29.186686 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:34:29.186686 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:34:29.186686 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 10:34:29.186686 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 10:34:29.186686 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 10:34:29.186686 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 10:34:29.187527 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 10:34:29.187527 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 10:34:29.187527 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 10:34:29.187527 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 10:34:29.187527 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:34:29.187527 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:34:29.187527 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:34:29.187527 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:34:29.187527 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:34:29.187527 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:34:29.187527 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:34:29.187527 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application with Lock( Jul 27 10:34:29.187527 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:34:29.187527 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:34:29.187527 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:34:29.187527 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:34:29.188308 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:34:29.188308 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:34:29.188308 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:34:29.188308 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:34:29.188308 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 10:34:29.188308 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 10:34:29.188308 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 10:34:29.188308 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 10:34:29.188308 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 10:34:29.188308 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 10:34:29.188308 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 27 10:34:29.188308 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Jul 27 10:34:29.188911 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 150/308] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jul 27 10:34:29 2022] GET /identity/v3/roles/reader => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 10:34:29.192523 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 159/309] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 10:34:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 27 10:34:29.198444 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3043b0-b16d-427e-a485-b6381b43a701 None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:29.203014 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:29.215255 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b136f08f-c870-4963-a131-a825654cc449 None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:29.264704 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:29.319878 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3043b0-b16d-427e-a485-b6381b43a701 None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:29.341856 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b136f08f-c870-4963-a131-a825654cc449 None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:29.364122 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de30f6c-cae4-48fe-b135-92a2a2042ffd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:29.364757 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de30f6c-cae4-48fe-b135-92a2a2042ffd None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:29.365039 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de30f6c-cae4-48fe-b135-92a2a2042ffd None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:29.365317 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de30f6c-cae4-48fe-b135-92a2a2042ffd None admin] PATH_INFO: `/v3/roles` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:29.366037 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de30f6c-cae4-48fe-b135-92a2a2042ffd None admin] RBAC: Adding query filter params (name=reader) {{(pid=86828) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:34:29.366342 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de30f6c-cae4-48fe-b135-92a2a2042ffd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:29.368294 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de30f6c-cae4-48fe-b135-92a2a2042ffd None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:29.384137 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 151/310] 10.222.0.36 () {56 vars in 1101 bytes} [Wed Jul 27 10:34:29 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:34:29.387590 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de30f6c-cae4-48fe-b135-92a2a2042ffd None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:29.397690 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca3da84-1c9e-410c-afdd-0d8ab3e7b2d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:29.400327 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca3da84-1c9e-410c-afdd-0d8ab3e7b2d2 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:29.400327 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca3da84-1c9e-410c-afdd-0d8ab3e7b2d2 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:29.400327 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca3da84-1c9e-410c-afdd-0d8ab3e7b2d2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:29.414593 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca3da84-1c9e-410c-afdd-0d8ab3e7b2d2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 10:34:29.422695 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application [None req-2ca3da84-1c9e-410c-afdd-0d8ab3e7b2d2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 27 10:34:29.422695 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:34:29.422695 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:34:29.422695 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:34:29.422695 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:34:29.422695 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:34:29.422695 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:34:29.422695 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:34:29.422695 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:34:29.422695 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:34:29.422695 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:34:29.422695 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:34:29.422695 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 10:34:29.422695 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 10:34:29.422695 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 10:34:29.422695 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 10:34:29.423559 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 10:34:29.423559 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 10:34:29.423559 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 10:34:29.423559 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 10:34:29.423559 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:34:29.423559 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:34:29.423559 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:34:29.423559 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:34:29.423559 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:34:29.423559 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:34:29.423559 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:34:29.423559 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application with Lock( Jul 27 10:34:29.423559 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:34:29.423559 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:34:29.423559 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:34:29.423559 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:34:29.424331 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:34:29.424331 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:34:29.424331 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:34:29.424331 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:34:29.424331 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 10:34:29.424331 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 10:34:29.424331 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 10:34:29.424331 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 10:34:29.424331 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 10:34:29.424331 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 10:34:29.424331 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 27 10:34:29.424331 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Jul 27 10:34:29.425921 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 160/311] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jul 27 10:34:29 2022] GET /identity/v3/roles/member => generated 84 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 10:34:29.432424 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca3da84-1c9e-410c-afdd-0d8ab3e7b2d2 None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:29.473168 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:29.519300 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de30f6c-cae4-48fe-b135-92a2a2042ffd None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:29.548155 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:29.552884 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca3da84-1c9e-410c-afdd-0d8ab3e7b2d2 None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:29.566038 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e078c46-8a2f-4838-ad0d-5d7ce44ca083 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:29.567756 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e078c46-8a2f-4838-ad0d-5d7ce44ca083 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:29.568269 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e078c46-8a2f-4838-ad0d-5d7ce44ca083 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:29.568746 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e078c46-8a2f-4838-ad0d-5d7ce44ca083 None admin] PATH_INFO: `/v3/users/9a8d5697b327417aa265bb38fb008a75` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:29.571173 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e078c46-8a2f-4838-ad0d-5d7ce44ca083 None admin] RBAC: Authorizing `identity:get_user(user_id=9a8d5697b327417aa265bb38fb008a75)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:29.573635 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e078c46-8a2f-4838-ad0d-5d7ce44ca083 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:29.576093 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 152/312] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Jul 27 10:34:29 2022] GET /identity/v3/users/9a8d5697b327417aa265bb38fb008a75 => generated 297 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:34:29.587700 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e078c46-8a2f-4838-ad0d-5d7ce44ca083 None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:29.600780 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455caafa-2e45-4552-8161-acd054430c46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:29.601493 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455caafa-2e45-4552-8161-acd054430c46 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:29.601773 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455caafa-2e45-4552-8161-acd054430c46 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:29.602337 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455caafa-2e45-4552-8161-acd054430c46 None admin] PATH_INFO: `/v3/roles` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:29.602860 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455caafa-2e45-4552-8161-acd054430c46 None admin] RBAC: Adding query filter params (name=member) {{(pid=86827) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:34:29.603183 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455caafa-2e45-4552-8161-acd054430c46 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:29.604671 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455caafa-2e45-4552-8161-acd054430c46 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:29.613323 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 161/313] 10.222.0.36 () {56 vars in 1101 bytes} [Wed Jul 27 10:34:29 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 10:34:29.619433 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455caafa-2e45-4552-8161-acd054430c46 None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:29.706293 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e078c46-8a2f-4838-ad0d-5d7ce44ca083 None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:29.740259 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455caafa-2e45-4552-8161-acd054430c46 None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:29.741616 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea88bc9d-83ba-4a4b-b6eb-e5fa2d6d220f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:29.742987 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea88bc9d-83ba-4a4b-b6eb-e5fa2d6d220f None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:29.743415 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea88bc9d-83ba-4a4b-b6eb-e5fa2d6d220f None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:29.743903 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea88bc9d-83ba-4a4b-b6eb-e5fa2d6d220f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:29.746836 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:29.782190 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f481b0-3733-4b28-81a6-2747d509d877 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:29.782928 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f481b0-3733-4b28-81a6-2747d509d877 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:29.783212 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f481b0-3733-4b28-81a6-2747d509d877 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:29.783488 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f481b0-3733-4b28-81a6-2747d509d877 None admin] PATH_INFO: `/v3/users/1224eb09d9ed4d728fcd0460454635b2` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:29.784786 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f481b0-3733-4b28-81a6-2747d509d877 None admin] RBAC: Authorizing `identity:get_user(user_id=1224eb09d9ed4d728fcd0460454635b2)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:29.786293 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f481b0-3733-4b28-81a6-2747d509d877 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:29.790645 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 162/314] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Jul 27 10:34:29 2022] GET /identity/v3/users/1224eb09d9ed4d728fcd0460454635b2 => generated 293 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:34:29.794862 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f481b0-3733-4b28-81a6-2747d509d877 None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:29.823763 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea88bc9d-83ba-4a4b-b6eb-e5fa2d6d220f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:29.825294 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea88bc9d-83ba-4a4b-b6eb-e5fa2d6d220f None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:29.836539 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:29.933216 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 153/315] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jul 27 10:34:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 10:34:29.938861 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea88bc9d-83ba-4a4b-b6eb-e5fa2d6d220f None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:29.940359 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f481b0-3733-4b28-81a6-2747d509d877 None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:29.980964 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4306e8-556f-4b43-95e0-7d419061c8f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:29.981797 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4306e8-556f-4b43-95e0-7d419061c8f5 None admin] REQUEST_METHOD: `PUT` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:29.982188 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4306e8-556f-4b43-95e0-7d419061c8f5 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:29.982673 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4306e8-556f-4b43-95e0-7d419061c8f5 None admin] PATH_INFO: `/v3/system/users/1224eb09d9ed4d728fcd0460454635b2/roles/4c9fc0b3baed441da2126be41cc6b04a` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:29.982798 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:29.985825 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4306e8-556f-4b43-95e0-7d419061c8f5 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=1224eb09d9ed4d728fcd0460454635b2, role_id=4c9fc0b3baed441da2126be41cc6b04a)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:29.987456 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4306e8-556f-4b43-95e0-7d419061c8f5 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:29.998357 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 163/316] 10.222.0.36 () {58 vars in 1334 bytes} [Wed Jul 27 10:34:29 2022] PUT /identity/v3/system/users/1224eb09d9ed4d728fcd0460454635b2/roles/4c9fc0b3baed441da2126be41cc6b04a => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:34:30.077122 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea88bc9d-83ba-4a4b-b6eb-e5fa2d6d220f None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:30.118501 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b1e1a6-345e-4ca4-9e2f-37c7d7e3f2f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:30.119199 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b1e1a6-345e-4ca4-9e2f-37c7d7e3f2f3 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:30.119476 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b1e1a6-345e-4ca4-9e2f-37c7d7e3f2f3 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:30.119751 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b1e1a6-345e-4ca4-9e2f-37c7d7e3f2f3 None admin] PATH_INFO: `/v3/projects/046cff0c48ee4df282871e4eda9cf939` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:30.121565 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b1e1a6-345e-4ca4-9e2f-37c7d7e3f2f3 None admin] RBAC: Authorizing `identity:get_project(project_id=046cff0c48ee4df282871e4eda9cf939)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:30.123353 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b1e1a6-345e-4ca4-9e2f-37c7d7e3f2f3 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:30.125278 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 154/317] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Jul 27 10:34:29 2022] GET /identity/v3/projects/046cff0c48ee4df282871e4eda9cf939 => generated 302 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 10:34:30.131561 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4306e8-556f-4b43-95e0-7d419061c8f5 None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:30.237919 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:30.245301 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4306e8-556f-4b43-95e0-7d419061c8f5 None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:30.285545 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78ed411-36f8-4e2a-b59e-b6045aeb0678 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:30.286509 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78ed411-36f8-4e2a-b59e-b6045aeb0678 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:30.287020 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78ed411-36f8-4e2a-b59e-b6045aeb0678 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:30.287514 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78ed411-36f8-4e2a-b59e-b6045aeb0678 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:30.288508 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78ed411-36f8-4e2a-b59e-b6045aeb0678 None admin] RBAC: Adding query filter params (role.id=80f67bfde4f5494e9c6d825aafb44e11, scope.project.id=046cff0c48ee4df282871e4eda9cf939, user.id=9a8d5697b327417aa265bb38fb008a75) {{(pid=86827) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:34:30.289578 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78ed411-36f8-4e2a-b59e-b6045aeb0678 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:30.291266 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78ed411-36f8-4e2a-b59e-b6045aeb0678 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:30.297797 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 164/318] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Jul 27 10:34:30 2022] GET /identity/v3/role_assignments?user.id=9a8d5697b327417aa265bb38fb008a75&scope.project.id=046cff0c48ee4df282871e4eda9cf939&role.id=80f67bfde4f5494e9c6d825aafb44e11 => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:34:30.965029 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c209533-1e03-4455-8617-5e8ab29d0fe0 None None] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:30.965500 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c209533-1e03-4455-8617-5e8ab29d0fe0 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:30.965572 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c209533-1e03-4455-8617-5e8ab29d0fe0 None None] PATH_INFO: `/` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:30.966987 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 155/319] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 10:34:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 10:34:30.972327 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d72fd2-c553-4541-93dd-77e0f1e528f8 None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:30.972621 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d72fd2-c553-4541-93dd-77e0f1e528f8 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:30.972901 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d72fd2-c553-4541-93dd-77e0f1e528f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:30.991150 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-93d72fd2-c553-4541-93dd-77e0f1e528f8 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:34:30.992830 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-93d72fd2-c553-4541-93dd-77e0f1e528f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:34:31.018080 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428888e5-b9c1-4eec-ab1f-487ec77fbc08 None None] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:31.018371 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428888e5-b9c1-4eec-ab1f-487ec77fbc08 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:31.018846 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428888e5-b9c1-4eec-ab1f-487ec77fbc08 None None] PATH_INFO: `/` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:31.020453 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 156/320] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 10:34:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 10:34:31.025422 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e7bb37-3236-4169-bd6b-8c90d863a866 None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:31.025711 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e7bb37-3236-4169-bd6b-8c90d863a866 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:31.025991 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e7bb37-3236-4169-bd6b-8c90d863a866 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:31.044144 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-b5e7bb37-3236-4169-bd6b-8c90d863a866 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:34:31.045334 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-b5e7bb37-3236-4169-bd6b-8c90d863a866 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:34:31.063710 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 165/321] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 10:34:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 10:34:31.085641 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d72fd2-c553-4541-93dd-77e0f1e528f8 None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:31.114687 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 157/322] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 10:34:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 27 10:34:31.137258 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e7bb37-3236-4169-bd6b-8c90d863a866 None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:31.169675 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:31.177209 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:31.191830 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d72fd2-c553-4541-93dd-77e0f1e528f8 None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:31.228539 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265f27de-3b35-4e71-974f-75705ca5d5e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:31.229186 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265f27de-3b35-4e71-974f-75705ca5d5e5 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:31.229565 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265f27de-3b35-4e71-974f-75705ca5d5e5 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:31.229748 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265f27de-3b35-4e71-974f-75705ca5d5e5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:31.230973 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265f27de-3b35-4e71-974f-75705ca5d5e5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:31.232420 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265f27de-3b35-4e71-974f-75705ca5d5e5 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:31.234180 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 166/323] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 10:34:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 10:34:31.239515 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265f27de-3b35-4e71-974f-75705ca5d5e5 None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:31.253228 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e7bb37-3236-4169-bd6b-8c90d863a866 None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:31.296048 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f79381-5a40-45fb-9ca5-7dda1ad2b8dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:31.296963 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f79381-5a40-45fb-9ca5-7dda1ad2b8dd None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:31.297246 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f79381-5a40-45fb-9ca5-7dda1ad2b8dd None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:31.297520 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f79381-5a40-45fb-9ca5-7dda1ad2b8dd None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:31.311719 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f79381-5a40-45fb-9ca5-7dda1ad2b8dd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 10:34:31.316760 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application [None req-e8f79381-5a40-45fb-9ca5-7dda1ad2b8dd None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 27 10:34:31.316760 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:34:31.316760 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:34:31.316760 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:34:31.316760 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:34:31.316760 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:34:31.316760 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:34:31.316760 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:34:31.316760 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:34:31.316760 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:34:31.316760 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:34:31.316760 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:34:31.316760 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 10:34:31.316760 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 10:34:31.316760 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 10:34:31.316760 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 10:34:31.317634 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 10:34:31.317634 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 10:34:31.317634 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 10:34:31.317634 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 10:34:31.317634 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:34:31.317634 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:34:31.317634 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:34:31.317634 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:34:31.317634 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:34:31.317634 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:34:31.317634 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:34:31.317634 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application with Lock( Jul 27 10:34:31.317634 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:34:31.317634 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:34:31.317634 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:34:31.317634 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:34:31.319334 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:34:31.319334 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:34:31.319334 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:34:31.319334 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:34:31.319334 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 10:34:31.319334 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 10:34:31.319334 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 10:34:31.319334 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 10:34:31.319334 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 10:34:31.319334 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 10:34:31.319334 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 27 10:34:31.319334 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Jul 27 10:34:31.319334 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 158/324] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jul 27 10:34:31 2022] GET /identity/v3/roles/reader => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 10:34:31.324136 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f79381-5a40-45fb-9ca5-7dda1ad2b8dd None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:31.345916 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265f27de-3b35-4e71-974f-75705ca5d5e5 None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:31.380795 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a38cf1-8d25-47b9-9dde-692fb45fadae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:31.381414 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a38cf1-8d25-47b9-9dde-692fb45fadae None admin] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:31.381697 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a38cf1-8d25-47b9-9dde-692fb45fadae None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:31.382220 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a38cf1-8d25-47b9-9dde-692fb45fadae None admin] PATH_INFO: `/v3/groups` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:31.382686 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a38cf1-8d25-47b9-9dde-692fb45fadae None admin] RBAC: Authorizing `identity:create_group()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:31.384116 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a38cf1-8d25-47b9-9dde-692fb45fadae None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:31.400996 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 167/325] 10.222.0.36 () {60 vars in 1134 bytes} [Wed Jul 27 10:34:31 2022] POST /identity/v3/groups => generated 226 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 10:34:31.427820 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:31.440817 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f79381-5a40-45fb-9ca5-7dda1ad2b8dd None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:31.491656 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268631b0-b6b2-4537-92a0-1651e572dbe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:31.493128 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268631b0-b6b2-4537-92a0-1651e572dbe4 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:31.493594 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268631b0-b6b2-4537-92a0-1651e572dbe4 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:31.494066 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268631b0-b6b2-4537-92a0-1651e572dbe4 None admin] PATH_INFO: `/v3/roles` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:31.495030 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268631b0-b6b2-4537-92a0-1651e572dbe4 None admin] RBAC: Adding query filter params (name=reader) {{(pid=86828) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:34:31.495516 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268631b0-b6b2-4537-92a0-1651e572dbe4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:31.497097 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268631b0-b6b2-4537-92a0-1651e572dbe4 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:31.507004 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 159/326] 10.222.0.36 () {56 vars in 1101 bytes} [Wed Jul 27 10:34:31 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 10:34:31.513571 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a38cf1-8d25-47b9-9dde-692fb45fadae None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:31.533691 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:31.631292 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a38cf1-8d25-47b9-9dde-692fb45fadae None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:31.670283 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6071db3-36f7-45da-9ec3-c27a12f3100a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:31.670981 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6071db3-36f7-45da-9ec3-c27a12f3100a None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:31.671263 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6071db3-36f7-45da-9ec3-c27a12f3100a None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:31.671540 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6071db3-36f7-45da-9ec3-c27a12f3100a None admin] PATH_INFO: `/v3/users/b740b641ba2545caa817805367676e8f` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:31.672735 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6071db3-36f7-45da-9ec3-c27a12f3100a None admin] RBAC: Authorizing `identity:get_user(user_id=b740b641ba2545caa817805367676e8f)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:31.674251 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6071db3-36f7-45da-9ec3-c27a12f3100a None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:31.676109 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 168/327] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Jul 27 10:34:31 2022] GET /identity/v3/users/b740b641ba2545caa817805367676e8f => generated 293 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:34:31.682577 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268631b0-b6b2-4537-92a0-1651e572dbe4 None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:31.816049 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268631b0-b6b2-4537-92a0-1651e572dbe4 None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:31.862052 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a838b0c-75d6-454c-8c1e-1aaed6add57a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:31.863193 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a838b0c-75d6-454c-8c1e-1aaed6add57a None admin] REQUEST_METHOD: `PUT` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:31.863678 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a838b0c-75d6-454c-8c1e-1aaed6add57a None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:31.864230 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a838b0c-75d6-454c-8c1e-1aaed6add57a None admin] PATH_INFO: `/v3/system/users/b740b641ba2545caa817805367676e8f/roles/80f67bfde4f5494e9c6d825aafb44e11` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:31.865931 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a838b0c-75d6-454c-8c1e-1aaed6add57a None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=b740b641ba2545caa817805367676e8f, role_id=80f67bfde4f5494e9c6d825aafb44e11)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:31.868129 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a838b0c-75d6-454c-8c1e-1aaed6add57a None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:31.880431 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 160/328] 10.222.0.36 () {58 vars in 1334 bytes} [Wed Jul 27 10:34:31 2022] PUT /identity/v3/system/users/b740b641ba2545caa817805367676e8f/roles/80f67bfde4f5494e9c6d825aafb44e11 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:34:32.078190 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0109fd55-f775-499d-8b64-8aff727c70fc None None] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:32.078377 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0109fd55-f775-499d-8b64-8aff727c70fc None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:32.078686 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0109fd55-f775-499d-8b64-8aff727c70fc None None] PATH_INFO: `/` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:32.080208 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 169/329] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 10:34:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 10:34:32.085803 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7729e1-d966-4ec9-89f5-f14ca62e315e None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:32.086708 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7729e1-d966-4ec9-89f5-f14ca62e315e None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:32.087201 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7729e1-d966-4ec9-89f5-f14ca62e315e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:32.107468 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-bf7729e1-d966-4ec9-89f5-f14ca62e315e None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:34:32.108980 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-bf7729e1-d966-4ec9-89f5-f14ca62e315e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:34:32.123579 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:32.185942 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 161/330] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 10:34:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 10:34:32.210531 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0109fd55-f775-499d-8b64-8aff727c70fc None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:32.317820 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:32.336183 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0109fd55-f775-499d-8b64-8aff727c70fc None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:32.375206 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe90373-323f-45ef-9208-3636ca4a5d28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:32.375855 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe90373-323f-45ef-9208-3636ca4a5d28 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:32.376134 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe90373-323f-45ef-9208-3636ca4a5d28 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:32.376513 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe90373-323f-45ef-9208-3636ca4a5d28 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:32.388179 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe90373-323f-45ef-9208-3636ca4a5d28 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 10:34:32.395948 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application [None req-bfe90373-323f-45ef-9208-3636ca4a5d28 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 27 10:34:32.395948 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:34:32.395948 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:34:32.395948 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:34:32.395948 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:34:32.395948 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:34:32.395948 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:34:32.395948 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:34:32.395948 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:34:32.395948 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:34:32.395948 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:34:32.395948 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:34:32.395948 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 10:34:32.395948 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 10:34:32.395948 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 10:34:32.395948 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 10:34:32.396866 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 10:34:32.396866 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 10:34:32.396866 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 10:34:32.396866 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 10:34:32.396866 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:34:32.396866 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:34:32.396866 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:34:32.396866 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:34:32.396866 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:34:32.396866 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:34:32.396866 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:34:32.396866 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application with Lock( Jul 27 10:34:32.396866 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:34:32.396866 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:34:32.396866 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:34:32.396866 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:34:32.397632 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:34:32.397632 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:34:32.397632 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:34:32.397632 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:34:32.397632 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 10:34:32.397632 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 10:34:32.397632 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 10:34:32.397632 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 10:34:32.397632 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 10:34:32.397632 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 10:34:32.397632 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 27 10:34:32.397632 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Jul 27 10:34:32.397632 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 170/331] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jul 27 10:34:32 2022] GET /identity/v3/roles/member => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 10:34:32.404553 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7729e1-d966-4ec9-89f5-f14ca62e315e None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:32.535893 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7729e1-d966-4ec9-89f5-f14ca62e315e None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:32.579484 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4f3f12-0fd6-4aed-b082-0ed1c08d32ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:32.580115 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4f3f12-0fd6-4aed-b082-0ed1c08d32ba None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:32.580393 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4f3f12-0fd6-4aed-b082-0ed1c08d32ba None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:32.580665 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4f3f12-0fd6-4aed-b082-0ed1c08d32ba None admin] PATH_INFO: `/v3/roles` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:32.581395 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4f3f12-0fd6-4aed-b082-0ed1c08d32ba None admin] RBAC: Adding query filter params (name=member) {{(pid=86828) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:34:32.581782 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4f3f12-0fd6-4aed-b082-0ed1c08d32ba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:32.583214 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4f3f12-0fd6-4aed-b082-0ed1c08d32ba None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:32.595321 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 162/332] 10.222.0.36 () {56 vars in 1101 bytes} [Wed Jul 27 10:34:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:34:32.600752 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe90373-323f-45ef-9208-3636ca4a5d28 None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:32.718942 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe90373-323f-45ef-9208-3636ca4a5d28 None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:32.756487 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a231ce-a20c-4c80-94c4-748eb151c522 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:32.757199 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a231ce-a20c-4c80-94c4-748eb151c522 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:32.757537 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a231ce-a20c-4c80-94c4-748eb151c522 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:32.757874 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a231ce-a20c-4c80-94c4-748eb151c522 None admin] PATH_INFO: `/v3/users/1224eb09d9ed4d728fcd0460454635b2` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:32.759124 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a231ce-a20c-4c80-94c4-748eb151c522 None admin] RBAC: Authorizing `identity:get_user(user_id=1224eb09d9ed4d728fcd0460454635b2)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:32.760842 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a231ce-a20c-4c80-94c4-748eb151c522 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:32.762812 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 171/333] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Jul 27 10:34:32 2022] GET /identity/v3/users/1224eb09d9ed4d728fcd0460454635b2 => generated 293 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:34:32.768600 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4f3f12-0fd6-4aed-b082-0ed1c08d32ba None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:32.782640 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:32.887749 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4f3f12-0fd6-4aed-b082-0ed1c08d32ba None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:32.927850 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c989968f-f244-46d4-ad9d-b47d0f7a7fc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:32.928653 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c989968f-f244-46d4-ad9d-b47d0f7a7fc1 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:32.928991 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c989968f-f244-46d4-ad9d-b47d0f7a7fc1 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:32.929289 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c989968f-f244-46d4-ad9d-b47d0f7a7fc1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:32.930901 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c989968f-f244-46d4-ad9d-b47d0f7a7fc1 None admin] RBAC: Adding query filter params (role.id=4c9fc0b3baed441da2126be41cc6b04a, user.id=1224eb09d9ed4d728fcd0460454635b2, scope.system=all) {{(pid=86828) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:34:32.931295 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c989968f-f244-46d4-ad9d-b47d0f7a7fc1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:32.933049 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c989968f-f244-46d4-ad9d-b47d0f7a7fc1 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:32.965808 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 163/334] 10.222.0.36 () {56 vars in 1308 bytes} [Wed Jul 27 10:34:32 2022] GET /identity/v3/role_assignments?user.id=1224eb09d9ed4d728fcd0460454635b2&scope.system=all&role.id=4c9fc0b3baed441da2126be41cc6b04a => generated 510 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:34:33.920659 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b829fd38-d692-43db-bfb1-d22d6111bef3 None None] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:33.921208 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b829fd38-d692-43db-bfb1-d22d6111bef3 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:33.921208 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b829fd38-d692-43db-bfb1-d22d6111bef3 None None] PATH_INFO: `/` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:33.922836 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 172/335] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 10:34:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 10:34:33.928603 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216082df-7ade-446d-8cf7-505a0f160f33 None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:33.929937 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216082df-7ade-446d-8cf7-505a0f160f33 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:33.930417 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216082df-7ade-446d-8cf7-505a0f160f33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:33.959643 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-216082df-7ade-446d-8cf7-505a0f160f33 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:34:33.961440 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-216082df-7ade-446d-8cf7-505a0f160f33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:34:34.048757 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:34.057530 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 164/336] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 10:34:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 10:34:34.084418 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b829fd38-d692-43db-bfb1-d22d6111bef3 None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:34.100466 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:34.198591 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b829fd38-d692-43db-bfb1-d22d6111bef3 None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:34.237005 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47e6897-6612-4e63-81f2-0813ed6683f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:34.237669 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47e6897-6612-4e63-81f2-0813ed6683f7 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:34.237956 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47e6897-6612-4e63-81f2-0813ed6683f7 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:34.238236 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47e6897-6612-4e63-81f2-0813ed6683f7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:34.249436 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47e6897-6612-4e63-81f2-0813ed6683f7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 10:34:34.257369 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application [None req-d47e6897-6612-4e63-81f2-0813ed6683f7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 27 10:34:34.257369 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:34:34.257369 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:34:34.257369 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:34:34.257369 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:34:34.257369 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:34:34.257369 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:34:34.257369 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:34:34.257369 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:34:34.257369 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:34:34.257369 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:34:34.257369 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:34:34.257369 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 10:34:34.257369 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 10:34:34.257369 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 10:34:34.257369 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 10:34:34.258313 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 10:34:34.258313 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 10:34:34.258313 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 10:34:34.258313 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 10:34:34.258313 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:34:34.258313 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:34:34.258313 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:34:34.258313 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:34:34.258313 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:34:34.258313 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:34:34.258313 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:34:34.258313 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application with Lock( Jul 27 10:34:34.258313 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:34:34.258313 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:34:34.258313 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:34:34.258313 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:34:34.259822 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:34:34.259822 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:34:34.259822 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:34:34.259822 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:34:34.259822 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 10:34:34.259822 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 10:34:34.259822 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 10:34:34.259822 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 10:34:34.259822 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 10:34:34.259822 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 10:34:34.259822 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 27 10:34:34.259822 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Jul 27 10:34:34.259822 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 173/337] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jul 27 10:34:34 2022] GET /identity/v3/roles/member => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 10:34:34.265171 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216082df-7ade-446d-8cf7-505a0f160f33 None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:34.378934 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216082df-7ade-446d-8cf7-505a0f160f33 None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:34.420957 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c7761c-7749-4fc8-92e0-de76072bfe7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:34.421598 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c7761c-7749-4fc8-92e0-de76072bfe7d None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:34.421883 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c7761c-7749-4fc8-92e0-de76072bfe7d None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:34.422162 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c7761c-7749-4fc8-92e0-de76072bfe7d None admin] PATH_INFO: `/v3/roles` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:34.422931 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c7761c-7749-4fc8-92e0-de76072bfe7d None admin] RBAC: Adding query filter params (name=member) {{(pid=86828) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:34:34.423244 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c7761c-7749-4fc8-92e0-de76072bfe7d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:34.424627 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c7761c-7749-4fc8-92e0-de76072bfe7d None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:34.434722 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 165/338] 10.222.0.36 () {56 vars in 1101 bytes} [Wed Jul 27 10:34:34 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:34:34.440483 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47e6897-6612-4e63-81f2-0813ed6683f7 None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:34.517177 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:34.549878 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47e6897-6612-4e63-81f2-0813ed6683f7 None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:34.587863 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cd01d6-d851-4577-93bc-0d2d6066b489 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:34.588507 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cd01d6-d851-4577-93bc-0d2d6066b489 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:34.588792 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cd01d6-d851-4577-93bc-0d2d6066b489 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:34.589076 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cd01d6-d851-4577-93bc-0d2d6066b489 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:34.661696 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6cd01d6-d851-4577-93bc-0d2d6066b489 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:34.663225 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6cd01d6-d851-4577-93bc-0d2d6066b489 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:34.742981 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:34.775153 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 174/339] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jul 27 10:34:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 10:34:34.780732 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c7761c-7749-4fc8-92e0-de76072bfe7d None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:34.844028 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:34.874224 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b097684-654f-414a-b472-bda77c92bc12 None None] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:34.874571 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b097684-654f-414a-b472-bda77c92bc12 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:34.874826 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b097684-654f-414a-b472-bda77c92bc12 None None] PATH_INFO: `/` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:34.876524 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 175/340] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 10:34:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 10:34:34.887948 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c923cc86-fe2a-4f1b-8e7b-51001df803a7 None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:34.888235 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c923cc86-fe2a-4f1b-8e7b-51001df803a7 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:34.888518 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c923cc86-fe2a-4f1b-8e7b-51001df803a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:34.898241 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c7761c-7749-4fc8-92e0-de76072bfe7d None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:34.934557 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-c923cc86-fe2a-4f1b-8e7b-51001df803a7 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:34:34.935805 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-c923cc86-fe2a-4f1b-8e7b-51001df803a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:34:34.951345 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5015608-06e9-4ca2-8699-df750221c70a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:34.954814 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5015608-06e9-4ca2-8699-df750221c70a None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:34.954814 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5015608-06e9-4ca2-8699-df750221c70a None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:34.954814 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5015608-06e9-4ca2-8699-df750221c70a None admin] PATH_INFO: `/v3/projects/33cf2502c8694374bd447d6304e658c2` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:34.959418 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5015608-06e9-4ca2-8699-df750221c70a None admin] RBAC: Authorizing `identity:get_project(project_id=33cf2502c8694374bd447d6304e658c2)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:34.961440 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5015608-06e9-4ca2-8699-df750221c70a None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:34.963215 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 166/341] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Jul 27 10:34:34 2022] GET /identity/v3/projects/33cf2502c8694374bd447d6304e658c2 => generated 298 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:34:34.969701 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5015608-06e9-4ca2-8699-df750221c70a None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:35.113287 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 176/342] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 10:34:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 10:34:35.114579 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b276705-3bd6-4341-8bbc-52cef4b1984b None None] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:35.114894 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b276705-3bd6-4341-8bbc-52cef4b1984b None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:35.115174 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b276705-3bd6-4341-8bbc-52cef4b1984b None None] PATH_INFO: `/` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:35.116431 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 177/343] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 10:34:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 10:34:35.128687 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc50efa7-3016-431e-893a-1afdcac38dfb None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:35.128687 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc50efa7-3016-431e-893a-1afdcac38dfb None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:35.128687 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc50efa7-3016-431e-893a-1afdcac38dfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:35.157619 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5015608-06e9-4ca2-8699-df750221c70a None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:35.159197 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-cc50efa7-3016-431e-893a-1afdcac38dfb None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:34:35.160487 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-cc50efa7-3016-431e-893a-1afdcac38dfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:34:35.187478 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:35.207370 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131917cd-b805-428c-a8e2-f5e6e6cee5fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:35.208041 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131917cd-b805-428c-a8e2-f5e6e6cee5fc None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:35.208326 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131917cd-b805-428c-a8e2-f5e6e6cee5fc None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:35.208604 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131917cd-b805-428c-a8e2-f5e6e6cee5fc None admin] PATH_INFO: `/v3/groups/56c1bbce28be4b57a92c54fae3188db0` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:35.223504 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131917cd-b805-428c-a8e2-f5e6e6cee5fc None admin] RBAC: Authorizing `identity:get_group(group_id=56c1bbce28be4b57a92c54fae3188db0)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:35.225726 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131917cd-b805-428c-a8e2-f5e6e6cee5fc None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:35.229167 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 167/344] 10.222.0.36 () {56 vars in 1180 bytes} [Wed Jul 27 10:34:34 2022] GET /identity/v3/groups/56c1bbce28be4b57a92c54fae3188db0 => generated 226 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 10:34:35.230028 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131917cd-b805-428c-a8e2-f5e6e6cee5fc None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:35.293627 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 178/345] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 10:34:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 10:34:35.294283 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc50efa7-3016-431e-893a-1afdcac38dfb None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:35.343320 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:35.398644 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131917cd-b805-428c-a8e2-f5e6e6cee5fc None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:35.427054 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc50efa7-3016-431e-893a-1afdcac38dfb None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:35.441571 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a4b2a4-1a59-4d4a-9a1a-08094a0037c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:35.441571 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a4b2a4-1a59-4d4a-9a1a-08094a0037c1 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:35.441571 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a4b2a4-1a59-4d4a-9a1a-08094a0037c1 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:35.441571 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a4b2a4-1a59-4d4a-9a1a-08094a0037c1 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:35.454593 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a4b2a4-1a59-4d4a-9a1a-08094a0037c1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 10:34:35.462891 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application [None req-26a4b2a4-1a59-4d4a-9a1a-08094a0037c1 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 27 10:34:35.462891 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:34:35.462891 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:34:35.462891 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:34:35.462891 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:34:35.462891 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:34:35.462891 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:34:35.462891 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:34:35.462891 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:34:35.462891 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:34:35.462891 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:34:35.462891 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:34:35.462891 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 10:34:35.462891 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 10:34:35.462891 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 10:34:35.462891 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 10:34:35.463777 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 10:34:35.463777 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 10:34:35.463777 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 10:34:35.463777 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 10:34:35.463777 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:34:35.463777 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:34:35.463777 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:34:35.463777 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:34:35.463777 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:34:35.463777 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:34:35.463777 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:34:35.463777 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application with Lock( Jul 27 10:34:35.463777 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:34:35.463777 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:34:35.463777 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:34:35.463777 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:34:35.464570 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:34:35.464570 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:34:35.464570 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:34:35.464570 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:34:35.464570 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 10:34:35.464570 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 10:34:35.464570 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 10:34:35.464570 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 10:34:35.464570 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 10:34:35.464570 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 10:34:35.464570 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 27 10:34:35.464570 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Jul 27 10:34:35.464570 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 168/346] 10.222.0.36 () {56 vars in 1114 bytes} [Wed Jul 27 10:34:35 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 10:34:35.465229 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a4b2a4-1a59-4d4a-9a1a-08094a0037c1 None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:35.478318 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b07ed1-4b4f-4d69-a5ee-72370d9b5bc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:35.479352 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b07ed1-4b4f-4d69-a5ee-72370d9b5bc1 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:35.479878 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b07ed1-4b4f-4d69-a5ee-72370d9b5bc1 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:35.480349 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b07ed1-4b4f-4d69-a5ee-72370d9b5bc1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:35.481474 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b07ed1-4b4f-4d69-a5ee-72370d9b5bc1 None admin] RBAC: Adding query filter params (group.id=56c1bbce28be4b57a92c54fae3188db0, role.id=4c9fc0b3baed441da2126be41cc6b04a, scope.project.id=33cf2502c8694374bd447d6304e658c2) {{(pid=86827) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:34:35.482049 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b07ed1-4b4f-4d69-a5ee-72370d9b5bc1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:35.483857 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b07ed1-4b4f-4d69-a5ee-72370d9b5bc1 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:35.491238 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 179/347] 10.222.0.36 () {56 vars in 1376 bytes} [Wed Jul 27 10:34:35 2022] GET /identity/v3/role_assignments?group.id=56c1bbce28be4b57a92c54fae3188db0&scope.project.id=33cf2502c8694374bd447d6304e658c2&role.id=4c9fc0b3baed441da2126be41cc6b04a => generated 260 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:34:35.493586 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01302a8-a0ab-49fc-b445-624d20b38f57 None None] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:35.494102 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01302a8-a0ab-49fc-b445-624d20b38f57 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:35.494583 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01302a8-a0ab-49fc-b445-624d20b38f57 None None] PATH_INFO: `/` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:35.496204 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 180/348] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 10:34:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 10:34:35.499493 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01302a8-a0ab-49fc-b445-624d20b38f57 None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:35.540589 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:35.653826 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a4b2a4-1a59-4d4a-9a1a-08094a0037c1 None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:35.658452 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:35.669302 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01302a8-a0ab-49fc-b445-624d20b38f57 None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:35.700627 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e31483c-4666-4b19-a6ad-589ab5009056 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:35.701282 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e31483c-4666-4b19-a6ad-589ab5009056 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:35.701568 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e31483c-4666-4b19-a6ad-589ab5009056 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:35.701847 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e31483c-4666-4b19-a6ad-589ab5009056 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:35.711339 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db22333c-484a-4d64-a076-8eb2da3dd4f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:35.712405 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db22333c-484a-4d64-a076-8eb2da3dd4f2 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:35.712985 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db22333c-484a-4d64-a076-8eb2da3dd4f2 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:35.713459 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db22333c-484a-4d64-a076-8eb2da3dd4f2 None admin] PATH_INFO: `/v3/roles` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:35.714449 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db22333c-484a-4d64-a076-8eb2da3dd4f2 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86827) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:34:35.714975 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db22333c-484a-4d64-a076-8eb2da3dd4f2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:35.715503 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e31483c-4666-4b19-a6ad-589ab5009056 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 10:34:35.716804 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db22333c-484a-4d64-a076-8eb2da3dd4f2 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:35.720589 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application [None req-3e31483c-4666-4b19-a6ad-589ab5009056 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 27 10:34:35.720589 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:34:35.720589 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:34:35.720589 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:34:35.720589 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:34:35.720589 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:34:35.720589 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:34:35.720589 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:34:35.720589 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:34:35.720589 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:34:35.720589 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:34:35.720589 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:34:35.720589 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 10:34:35.720589 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 10:34:35.720589 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 10:34:35.720589 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 10:34:35.721583 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 10:34:35.721583 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 10:34:35.721583 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 10:34:35.721583 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 10:34:35.721583 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:34:35.721583 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:34:35.721583 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:34:35.721583 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:34:35.721583 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:34:35.721583 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:34:35.721583 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:34:35.721583 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application with Lock( Jul 27 10:34:35.721583 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:34:35.721583 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:34:35.721583 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:34:35.721583 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:34:35.722851 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:34:35.722851 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:34:35.722851 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:34:35.722851 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:34:35.722851 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 10:34:35.722851 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 10:34:35.722851 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 10:34:35.722851 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 10:34:35.722851 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 10:34:35.722851 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 10:34:35.722851 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 27 10:34:35.722851 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Jul 27 10:34:35.722851 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 169/349] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jul 27 10:34:35 2022] GET /identity/v3/roles/reader => generated 84 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 10:34:35.723765 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5bf904-35d5-4723-850c-357934b3c020 None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:35.723838 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5bf904-35d5-4723-850c-357934b3c020 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:35.724094 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5bf904-35d5-4723-850c-357934b3c020 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:35.732838 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 181/350] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Jul 27 10:34:35 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:34:35.734205 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db22333c-484a-4d64-a076-8eb2da3dd4f2 None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:35.745783 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-7c5bf904-35d5-4723-850c-357934b3c020 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:34:35.747047 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-7c5bf904-35d5-4723-850c-357934b3c020 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:34:35.881634 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 170/351] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 10:34:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 10:34:35.883712 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5bf904-35d5-4723-850c-357934b3c020 None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:35.928530 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db22333c-484a-4d64-a076-8eb2da3dd4f2 None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:35.980056 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:36.008892 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece1aaea-c320-44d6-b25b-4632e031e60b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:36.011246 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece1aaea-c320-44d6-b25b-4632e031e60b None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:36.011897 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece1aaea-c320-44d6-b25b-4632e031e60b None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:36.013591 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece1aaea-c320-44d6-b25b-4632e031e60b None admin] PATH_INFO: `/v3/roles` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:36.015151 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece1aaea-c320-44d6-b25b-4632e031e60b None admin] RBAC: Adding query filter params (name=reader) {{(pid=86827) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:34:36.015877 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece1aaea-c320-44d6-b25b-4632e031e60b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:36.018102 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece1aaea-c320-44d6-b25b-4632e031e60b None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:36.038582 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 182/352] 10.222.0.36 () {56 vars in 1101 bytes} [Wed Jul 27 10:34:35 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:34:36.039801 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece1aaea-c320-44d6-b25b-4632e031e60b None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:36.084360 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:36.095924 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5bf904-35d5-4723-850c-357934b3c020 None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:36.149513 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217ce444-4309-4cf2-8b4b-a58348eac971 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:36.150516 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217ce444-4309-4cf2-8b4b-a58348eac971 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:36.151028 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217ce444-4309-4cf2-8b4b-a58348eac971 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:36.151521 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217ce444-4309-4cf2-8b4b-a58348eac971 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:36.246063 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece1aaea-c320-44d6-b25b-4632e031e60b None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:36.282574 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217ce444-4309-4cf2-8b4b-a58348eac971 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:36.284275 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217ce444-4309-4cf2-8b4b-a58348eac971 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:36.296676 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2672b4-9b5e-4c6a-85ec-7b0dc1370bcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:36.297780 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2672b4-9b5e-4c6a-85ec-7b0dc1370bcf None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:36.298201 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2672b4-9b5e-4c6a-85ec-7b0dc1370bcf None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:36.298481 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2672b4-9b5e-4c6a-85ec-7b0dc1370bcf None admin] PATH_INFO: `/v3/roles/member` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:36.312120 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2672b4-9b5e-4c6a-85ec-7b0dc1370bcf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 10:34:36.319022 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application [None req-ee2672b4-9b5e-4c6a-85ec-7b0dc1370bcf None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 27 10:34:36.319022 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:34:36.319022 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:34:36.319022 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:34:36.319022 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:34:36.319022 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:34:36.319022 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:34:36.319022 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:34:36.319022 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:34:36.319022 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:34:36.319022 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:34:36.319022 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:34:36.319022 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 10:34:36.319022 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 10:34:36.319022 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 10:34:36.319022 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 10:34:36.319946 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 10:34:36.319946 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 10:34:36.319946 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 10:34:36.319946 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 10:34:36.319946 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:34:36.319946 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:34:36.319946 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:34:36.319946 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:34:36.319946 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:34:36.319946 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:34:36.319946 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:34:36.319946 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application with Lock( Jul 27 10:34:36.319946 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:34:36.319946 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:34:36.319946 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:34:36.319946 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:34:36.321311 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:34:36.321311 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:34:36.321311 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:34:36.321311 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:34:36.321311 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 10:34:36.321311 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 10:34:36.321311 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 10:34:36.321311 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 10:34:36.321311 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 10:34:36.321311 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 10:34:36.321311 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 27 10:34:36.321311 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Jul 27 10:34:36.322654 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 183/353] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jul 27 10:34:36 2022] GET /identity/v3/roles/member => generated 84 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 10:34:36.322654 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2672b4-9b5e-4c6a-85ec-7b0dc1370bcf None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:36.336475 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:36.412504 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 171/354] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jul 27 10:34:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 530 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 10:34:36.413914 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510e55ed-d790-47a3-acf2-0478339f6f8e None None] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:36.414217 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510e55ed-d790-47a3-acf2-0478339f6f8e None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:36.414500 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510e55ed-d790-47a3-acf2-0478339f6f8e None None] PATH_INFO: `/` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:36.416203 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 172/355] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 10:34:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 10:34:36.416710 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510e55ed-d790-47a3-acf2-0478339f6f8e None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:36.428272 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:36.454940 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2672b4-9b5e-4c6a-85ec-7b0dc1370bcf None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:36.500288 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551d2d5f-115a-4eaa-b910-7d52122ecf04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:36.501061 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551d2d5f-115a-4eaa-b910-7d52122ecf04 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:36.501510 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551d2d5f-115a-4eaa-b910-7d52122ecf04 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:36.501799 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551d2d5f-115a-4eaa-b910-7d52122ecf04 None admin] PATH_INFO: `/v3/users/b740b641ba2545caa817805367676e8f` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:36.503032 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551d2d5f-115a-4eaa-b910-7d52122ecf04 None admin] RBAC: Authorizing `identity:get_user(user_id=b740b641ba2545caa817805367676e8f)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:36.504665 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551d2d5f-115a-4eaa-b910-7d52122ecf04 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:36.507160 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 184/356] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Jul 27 10:34:36 2022] GET /identity/v3/users/b740b641ba2545caa817805367676e8f => generated 293 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:34:36.508163 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551d2d5f-115a-4eaa-b910-7d52122ecf04 None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:36.560013 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510e55ed-d790-47a3-acf2-0478339f6f8e None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:36.613174 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d699d250-f028-47ad-8769-2ab3a7694669 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:36.613174 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d699d250-f028-47ad-8769-2ab3a7694669 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:36.613174 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d699d250-f028-47ad-8769-2ab3a7694669 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:36.613174 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d699d250-f028-47ad-8769-2ab3a7694669 None admin] PATH_INFO: `/v3/roles` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:36.613570 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d699d250-f028-47ad-8769-2ab3a7694669 None admin] RBAC: Adding query filter params (name=member) {{(pid=86828) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:34:36.613570 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d699d250-f028-47ad-8769-2ab3a7694669 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:36.614178 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d699d250-f028-47ad-8769-2ab3a7694669 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:36.626366 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 173/357] 10.222.0.36 () {56 vars in 1101 bytes} [Wed Jul 27 10:34:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:34:36.628289 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ac9e85-aa77-48b2-b274-6d38046e4eaa None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:36.628618 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ac9e85-aa77-48b2-b274-6d38046e4eaa None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:36.628898 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ac9e85-aa77-48b2-b274-6d38046e4eaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:36.638852 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:36.652417 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-96ac9e85-aa77-48b2-b274-6d38046e4eaa None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:34:36.653798 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-96ac9e85-aa77-48b2-b274-6d38046e4eaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:34:36.658069 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551d2d5f-115a-4eaa-b910-7d52122ecf04 None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:36.704820 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846a3bbc-8614-4d80-9058-46cd419b3df5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:36.706294 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846a3bbc-8614-4d80-9058-46cd419b3df5 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:36.706583 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846a3bbc-8614-4d80-9058-46cd419b3df5 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:36.706902 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846a3bbc-8614-4d80-9058-46cd419b3df5 None admin] PATH_INFO: `/v3/projects/33cf2502c8694374bd447d6304e658c2` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:36.708105 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846a3bbc-8614-4d80-9058-46cd419b3df5 None admin] RBAC: Authorizing `identity:get_project(project_id=33cf2502c8694374bd447d6304e658c2)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:36.709579 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846a3bbc-8614-4d80-9058-46cd419b3df5 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:36.711989 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 185/358] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Jul 27 10:34:36 2022] GET /identity/v3/projects/33cf2502c8694374bd447d6304e658c2 => generated 298 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:34:36.713384 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be69d581-add0-4c9f-b3bf-27b1f6aa46d7 None None] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:36.713671 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be69d581-add0-4c9f-b3bf-27b1f6aa46d7 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:36.713951 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be69d581-add0-4c9f-b3bf-27b1f6aa46d7 None None] PATH_INFO: `/` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:36.715759 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 186/359] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 10:34:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 10:34:36.716916 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be69d581-add0-4c9f-b3bf-27b1f6aa46d7 None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:36.738022 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:36.745457 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 174/360] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 10:34:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 10:34:36.746540 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ac9e85-aa77-48b2-b274-6d38046e4eaa None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:36.843680 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be69d581-add0-4c9f-b3bf-27b1f6aa46d7 None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:36.866310 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ac9e85-aa77-48b2-b274-6d38046e4eaa None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:36.888567 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5e733e-47dc-4371-b0e5-25bd135a1f06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:36.889517 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5e733e-47dc-4371-b0e5-25bd135a1f06 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:36.890032 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5e733e-47dc-4371-b0e5-25bd135a1f06 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:36.890522 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5e733e-47dc-4371-b0e5-25bd135a1f06 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:36.891581 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5e733e-47dc-4371-b0e5-25bd135a1f06 None admin] RBAC: Adding query filter params (role.id=80f67bfde4f5494e9c6d825aafb44e11, user.id=b740b641ba2545caa817805367676e8f, scope.system=all) {{(pid=86827) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:34:36.892107 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5e733e-47dc-4371-b0e5-25bd135a1f06 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:36.894788 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5e733e-47dc-4371-b0e5-25bd135a1f06 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:36.911319 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 187/361] 10.222.0.36 () {56 vars in 1308 bytes} [Wed Jul 27 10:34:36 2022] GET /identity/v3/role_assignments?user.id=b740b641ba2545caa817805367676e8f&scope.system=all&role.id=80f67bfde4f5494e9c6d825aafb44e11 => generated 510 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:34:36.912601 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5800a30a-bca0-453d-bb1f-e2e684cb7caf None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:36.914041 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5800a30a-bca0-453d-bb1f-e2e684cb7caf None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:36.914326 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5800a30a-bca0-453d-bb1f-e2e684cb7caf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:36.917291 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444d6942-a97c-4ef3-a2c8-d4cd41516227 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:36.918192 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444d6942-a97c-4ef3-a2c8-d4cd41516227 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:36.918500 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444d6942-a97c-4ef3-a2c8-d4cd41516227 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:36.918867 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444d6942-a97c-4ef3-a2c8-d4cd41516227 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:36.922468 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:36.955699 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-5800a30a-bca0-453d-bb1f-e2e684cb7caf None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:34:36.957017 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-5800a30a-bca0-453d-bb1f-e2e684cb7caf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:34:37.015543 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444d6942-a97c-4ef3-a2c8-d4cd41516227 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:37.017487 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444d6942-a97c-4ef3-a2c8-d4cd41516227 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:37.050720 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 188/362] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 10:34:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 10:34:37.051684 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5800a30a-bca0-453d-bb1f-e2e684cb7caf None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:37.066981 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:37.174185 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 175/363] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jul 27 10:34:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 10:34:37.176257 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444d6942-a97c-4ef3-a2c8-d4cd41516227 None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:37.207707 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5800a30a-bca0-453d-bb1f-e2e684cb7caf None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:37.223786 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:37.251516 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2117fafd-6abc-4e81-96b1-535a1679fc2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:37.252632 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2117fafd-6abc-4e81-96b1-535a1679fc2d None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:37.253395 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2117fafd-6abc-4e81-96b1-535a1679fc2d None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:37.253687 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2117fafd-6abc-4e81-96b1-535a1679fc2d None admin] PATH_INFO: `/v3/groups/56c1bbce28be4b57a92c54fae3188db0` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:37.255149 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2117fafd-6abc-4e81-96b1-535a1679fc2d None admin] RBAC: Authorizing `identity:get_group(group_id=56c1bbce28be4b57a92c54fae3188db0)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:37.257195 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2117fafd-6abc-4e81-96b1-535a1679fc2d None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:37.260736 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 189/364] 10.222.0.36 () {56 vars in 1180 bytes} [Wed Jul 27 10:34:37 2022] GET /identity/v3/groups/56c1bbce28be4b57a92c54fae3188db0 => generated 226 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:34:37.261940 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2117fafd-6abc-4e81-96b1-535a1679fc2d None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:37.310042 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444d6942-a97c-4ef3-a2c8-d4cd41516227 None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:37.347946 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:37.354564 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb857cd-81a8-470e-b8cb-98f9cc4284b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:37.355356 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb857cd-81a8-470e-b8cb-98f9cc4284b7 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:37.355714 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb857cd-81a8-470e-b8cb-98f9cc4284b7 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:37.356100 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb857cd-81a8-470e-b8cb-98f9cc4284b7 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:37.369770 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb857cd-81a8-470e-b8cb-98f9cc4284b7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 10:34:37.377181 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application [None req-7bb857cd-81a8-470e-b8cb-98f9cc4284b7 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 27 10:34:37.377181 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:34:37.377181 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:34:37.377181 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:34:37.377181 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:34:37.377181 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:34:37.377181 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:34:37.377181 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:34:37.377181 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:34:37.377181 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:34:37.377181 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:34:37.377181 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:34:37.377181 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 10:34:37.377181 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 10:34:37.377181 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 10:34:37.377181 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 10:34:37.378079 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 10:34:37.378079 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 10:34:37.378079 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 10:34:37.378079 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 10:34:37.378079 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:34:37.378079 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:34:37.378079 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:34:37.378079 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:34:37.378079 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:34:37.378079 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:34:37.378079 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:34:37.378079 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application with Lock( Jul 27 10:34:37.378079 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:34:37.378079 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:34:37.378079 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:34:37.378079 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:34:37.378894 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:34:37.378894 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:34:37.378894 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:34:37.378894 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:34:37.378894 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 10:34:37.378894 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 10:34:37.378894 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 10:34:37.378894 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 10:34:37.378894 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 10:34:37.378894 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 10:34:37.378894 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 27 10:34:37.378894 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Jul 27 10:34:37.378894 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 176/365] 10.222.0.36 () {56 vars in 1114 bytes} [Wed Jul 27 10:34:37 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 10:34:37.378894 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb857cd-81a8-470e-b8cb-98f9cc4284b7 None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:37.393534 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2117fafd-6abc-4e81-96b1-535a1679fc2d None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:37.437786 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb613cd-3941-4d12-837f-9b47286875e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:37.438466 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb613cd-3941-4d12-837f-9b47286875e2 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:37.438779 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb613cd-3941-4d12-837f-9b47286875e2 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:37.439062 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb613cd-3941-4d12-837f-9b47286875e2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:37.452939 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb613cd-3941-4d12-837f-9b47286875e2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 10:34:37.457935 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application [None req-bbb613cd-3941-4d12-837f-9b47286875e2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 10:34:37.457935 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:34:37.457935 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:34:37.457935 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:34:37.457935 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:34:37.457935 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:34:37.457935 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:34:37.457935 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:34:37.457935 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:34:37.457935 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:34:37.457935 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:34:37.457935 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:34:37.457935 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 10:34:37.457935 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 10:34:37.457935 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 10:34:37.457935 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 10:34:37.458934 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 10:34:37.458934 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 10:34:37.458934 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 10:34:37.458934 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 10:34:37.458934 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:34:37.458934 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:34:37.458934 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:34:37.458934 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:34:37.458934 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:34:37.458934 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:34:37.458934 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:34:37.458934 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application with Lock( Jul 27 10:34:37.458934 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:34:37.458934 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:34:37.458934 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:34:37.458934 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:34:37.459702 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:34:37.459702 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:34:37.459702 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:34:37.459702 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:34:37.459702 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 10:34:37.459702 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 10:34:37.459702 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 10:34:37.459702 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 10:34:37.459702 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 10:34:37.459702 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 10:34:37.459702 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 10:34:37.459702 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Jul 27 10:34:37.460293 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 190/366] 10.222.0.36 () {56 vars in 1096 bytes} [Wed Jul 27 10:34:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 10:34:37.465656 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb613cd-3941-4d12-837f-9b47286875e2 None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:37.503732 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:37.516742 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb857cd-81a8-470e-b8cb-98f9cc4284b7 None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:37.563778 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740b23d7-39e3-4518-98c4-6bc7e4aa734b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:37.564438 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740b23d7-39e3-4518-98c4-6bc7e4aa734b None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:37.564727 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740b23d7-39e3-4518-98c4-6bc7e4aa734b None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:37.565008 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740b23d7-39e3-4518-98c4-6bc7e4aa734b None admin] PATH_INFO: `/v3/projects/046cff0c48ee4df282871e4eda9cf939` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:37.567235 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740b23d7-39e3-4518-98c4-6bc7e4aa734b None admin] RBAC: Authorizing `identity:get_project(project_id=046cff0c48ee4df282871e4eda9cf939)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:37.569172 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740b23d7-39e3-4518-98c4-6bc7e4aa734b None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:37.572033 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 177/367] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Jul 27 10:34:37 2022] GET /identity/v3/projects/046cff0c48ee4df282871e4eda9cf939 => generated 302 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:34:37.572622 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740b23d7-39e3-4518-98c4-6bc7e4aa734b None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:37.594312 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb613cd-3941-4d12-837f-9b47286875e2 None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:37.631718 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:37.642249 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adb7d3c-ea56-44d1-b42d-be65bd70735a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:37.643040 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adb7d3c-ea56-44d1-b42d-be65bd70735a None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:37.643485 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adb7d3c-ea56-44d1-b42d-be65bd70735a None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:37.643770 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adb7d3c-ea56-44d1-b42d-be65bd70735a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:37.644716 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4adb7d3c-ea56-44d1-b42d-be65bd70735a None admin] RBAC: Adding query filter params (group.id=56c1bbce28be4b57a92c54fae3188db0, role.id=c3341f95596344309327efff99a20d5c, scope.project.id=33cf2502c8694374bd447d6304e658c2) {{(pid=86827) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:34:37.645033 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4adb7d3c-ea56-44d1-b42d-be65bd70735a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:37.646703 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4adb7d3c-ea56-44d1-b42d-be65bd70735a None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:37.653406 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 191/368] 10.222.0.36 () {56 vars in 1376 bytes} [Wed Jul 27 10:34:37 2022] GET /identity/v3/role_assignments?group.id=56c1bbce28be4b57a92c54fae3188db0&scope.project.id=33cf2502c8694374bd447d6304e658c2&role.id=c3341f95596344309327efff99a20d5c => generated 260 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:34:37.654358 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adb7d3c-ea56-44d1-b42d-be65bd70735a None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:37.699420 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740b23d7-39e3-4518-98c4-6bc7e4aa734b None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:37.744579 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3579f1-6b00-47eb-8894-354d9329c3f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:37.745347 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3579f1-6b00-47eb-8894-354d9329c3f9 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:37.745734 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3579f1-6b00-47eb-8894-354d9329c3f9 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:37.746148 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3579f1-6b00-47eb-8894-354d9329c3f9 None admin] PATH_INFO: `/v3/roles` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:37.746927 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3579f1-6b00-47eb-8894-354d9329c3f9 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86828) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:34:37.747237 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3579f1-6b00-47eb-8894-354d9329c3f9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:37.748746 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3579f1-6b00-47eb-8894-354d9329c3f9 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:37.761762 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 178/369] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Jul 27 10:34:37 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:34:37.762656 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3579f1-6b00-47eb-8894-354d9329c3f9 None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:37.800943 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:37.808234 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adb7d3c-ea56-44d1-b42d-be65bd70735a None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:37.849122 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9b545a-cc93-4a0b-ad5f-df3bf2c8648f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:37.849930 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9b545a-cc93-4a0b-ad5f-df3bf2c8648f None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:37.850243 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9b545a-cc93-4a0b-ad5f-df3bf2c8648f None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:37.850581 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9b545a-cc93-4a0b-ad5f-df3bf2c8648f None admin] PATH_INFO: `/v3/roles` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:37.851486 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9b545a-cc93-4a0b-ad5f-df3bf2c8648f None admin] RBAC: Adding query filter params (name=admin) {{(pid=86827) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:34:37.851869 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9b545a-cc93-4a0b-ad5f-df3bf2c8648f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:37.853493 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9b545a-cc93-4a0b-ad5f-df3bf2c8648f None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:37.862806 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 192/370] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jul 27 10:34:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:34:37.863656 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9b545a-cc93-4a0b-ad5f-df3bf2c8648f None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:37.908955 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3579f1-6b00-47eb-8894-354d9329c3f9 None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:37.944745 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:37.954193 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567b02fe-454c-49ef-9234-724d339cedc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:37.954823 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567b02fe-454c-49ef-9234-724d339cedc4 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:37.955108 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567b02fe-454c-49ef-9234-724d339cedc4 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:37.955384 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567b02fe-454c-49ef-9234-724d339cedc4 None admin] PATH_INFO: `/v3/groups/56c1bbce28be4b57a92c54fae3188db0` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:37.956542 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567b02fe-454c-49ef-9234-724d339cedc4 None admin] RBAC: Authorizing `identity:get_group(group_id=56c1bbce28be4b57a92c54fae3188db0)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:37.957989 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567b02fe-454c-49ef-9234-724d339cedc4 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:37.959806 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 179/371] 10.222.0.36 () {56 vars in 1180 bytes} [Wed Jul 27 10:34:37 2022] GET /identity/v3/groups/56c1bbce28be4b57a92c54fae3188db0 => generated 226 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:34:37.961737 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9865f3-92e1-4849-b79a-7e214df89b05 None None] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:37.961737 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9865f3-92e1-4849-b79a-7e214df89b05 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:37.961737 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9865f3-92e1-4849-b79a-7e214df89b05 None None] PATH_INFO: `/` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:37.962841 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 180/372] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 10:34:37 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 10:34:37.967016 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9865f3-92e1-4849-b79a-7e214df89b05 None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:38.012141 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9b545a-cc93-4a0b-ad5f-df3bf2c8648f None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:38.052844 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76d7495-507f-4221-a9f8-c8e0b3a148f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:38.053986 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76d7495-507f-4221-a9f8-c8e0b3a148f3 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:38.054826 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76d7495-507f-4221-a9f8-c8e0b3a148f3 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:38.055270 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76d7495-507f-4221-a9f8-c8e0b3a148f3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:38.086361 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:38.100017 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9865f3-92e1-4849-b79a-7e214df89b05 None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:38.149087 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161df71a-e55e-4838-ad11-0efed491ce89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:38.149828 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161df71a-e55e-4838-ad11-0efed491ce89 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:38.150182 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161df71a-e55e-4838-ad11-0efed491ce89 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:38.150532 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161df71a-e55e-4838-ad11-0efed491ce89 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:38.165520 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76d7495-507f-4221-a9f8-c8e0b3a148f3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:38.167103 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76d7495-507f-4221-a9f8-c8e0b3a148f3 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:38.236204 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161df71a-e55e-4838-ad11-0efed491ce89 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:38.237821 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:38.238089 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161df71a-e55e-4838-ad11-0efed491ce89 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:38.297958 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 193/373] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jul 27 10:34:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 10:34:38.297958 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76d7495-507f-4221-a9f8-c8e0b3a148f3 None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:38.355065 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:38.366189 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 181/374] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jul 27 10:34:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 10:34:38.370777 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbe9587-8b56-4b21-8d72-4c73cb85f6b4 None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:38.370777 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbe9587-8b56-4b21-8d72-4c73cb85f6b4 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:38.370777 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbe9587-8b56-4b21-8d72-4c73cb85f6b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:38.388999 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-6bbe9587-8b56-4b21-8d72-4c73cb85f6b4 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:34:38.390198 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-6bbe9587-8b56-4b21-8d72-4c73cb85f6b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:34:38.420271 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76d7495-507f-4221-a9f8-c8e0b3a148f3 None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:38.469060 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810026c8-8337-425e-8490-5365dcea6e28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:38.469693 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810026c8-8337-425e-8490-5365dcea6e28 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:38.469972 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810026c8-8337-425e-8490-5365dcea6e28 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:38.470246 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810026c8-8337-425e-8490-5365dcea6e28 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:38.471078 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810026c8-8337-425e-8490-5365dcea6e28 None admin] RBAC: Adding query filter params (group.id=56c1bbce28be4b57a92c54fae3188db0, role.id=4c9fc0b3baed441da2126be41cc6b04a, scope.project.id=046cff0c48ee4df282871e4eda9cf939) {{(pid=86827) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:34:38.471391 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810026c8-8337-425e-8490-5365dcea6e28 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:38.472848 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810026c8-8337-425e-8490-5365dcea6e28 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:38.475717 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 182/375] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 10:34:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 10:34:38.476867 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbe9587-8b56-4b21-8d72-4c73cb85f6b4 None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:38.485785 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 194/376] 10.222.0.36 () {56 vars in 1376 bytes} [Wed Jul 27 10:34:38 2022] GET /identity/v3/role_assignments?group.id=56c1bbce28be4b57a92c54fae3188db0&scope.project.id=046cff0c48ee4df282871e4eda9cf939&role.id=4c9fc0b3baed441da2126be41cc6b04a => generated 260 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:34:38.486488 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810026c8-8337-425e-8490-5365dcea6e28 None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:38.544842 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:38.599069 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbe9587-8b56-4b21-8d72-4c73cb85f6b4 None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:38.626339 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:38.641487 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcaba0a-ebe1-4f7e-bde8-8f08be79f0b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:38.642549 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcaba0a-ebe1-4f7e-bde8-8f08be79f0b0 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:38.643120 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcaba0a-ebe1-4f7e-bde8-8f08be79f0b0 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:38.643700 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcaba0a-ebe1-4f7e-bde8-8f08be79f0b0 None admin] PATH_INFO: `/v3/projects/046cff0c48ee4df282871e4eda9cf939` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:38.645153 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dcaba0a-ebe1-4f7e-bde8-8f08be79f0b0 None admin] RBAC: Authorizing `identity:get_project(project_id=046cff0c48ee4df282871e4eda9cf939)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:38.646489 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810026c8-8337-425e-8490-5365dcea6e28 None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:38.647122 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dcaba0a-ebe1-4f7e-bde8-8f08be79f0b0 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:38.649595 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 183/377] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Jul 27 10:34:38 2022] GET /identity/v3/projects/046cff0c48ee4df282871e4eda9cf939 => generated 302 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:34:38.650558 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcaba0a-ebe1-4f7e-bde8-8f08be79f0b0 None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:38.686105 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b6769b-7030-42b8-9788-a68f26248ee2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:38.686973 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b6769b-7030-42b8-9788-a68f26248ee2 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:38.687309 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b6769b-7030-42b8-9788-a68f26248ee2 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:38.687633 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b6769b-7030-42b8-9788-a68f26248ee2 None admin] PATH_INFO: `/v3/projects/eeee704de4024dd1b602d1a289cba407` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:38.688959 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b6769b-7030-42b8-9788-a68f26248ee2 None admin] RBAC: Authorizing `identity:get_project(project_id=eeee704de4024dd1b602d1a289cba407)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:38.690526 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b6769b-7030-42b8-9788-a68f26248ee2 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:38.697665 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 195/378] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Jul 27 10:34:38 2022] GET /identity/v3/projects/eeee704de4024dd1b602d1a289cba407 => generated 344 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:34:38.698843 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b6769b-7030-42b8-9788-a68f26248ee2 None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:38.789383 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcaba0a-ebe1-4f7e-bde8-8f08be79f0b0 None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:38.806770 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:38.818810 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b6769b-7030-42b8-9788-a68f26248ee2 None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:38.837715 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f6b902-cee6-4ad0-b1c3-ae5ed1aef641 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:38.838690 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f6b902-cee6-4ad0-b1c3-ae5ed1aef641 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:38.838983 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f6b902-cee6-4ad0-b1c3-ae5ed1aef641 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:38.839260 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f6b902-cee6-4ad0-b1c3-ae5ed1aef641 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:38.854751 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f6b902-cee6-4ad0-b1c3-ae5ed1aef641 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 10:34:38.859847 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6109daa-d49f-4604-aca2-661bee9b7e8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:38.860418 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application [None req-75f6b902-cee6-4ad0-b1c3-ae5ed1aef641 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 27 10:34:38.860418 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:34:38.860418 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:34:38.860418 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:34:38.860418 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:34:38.860418 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:34:38.860418 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:34:38.860418 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:34:38.860418 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:34:38.860418 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:34:38.860418 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:34:38.860418 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:34:38.860418 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 10:34:38.860418 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 10:34:38.860418 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 10:34:38.860418 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 10:34:38.861263 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 10:34:38.861263 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 10:34:38.861263 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 10:34:38.861263 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 10:34:38.861263 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:34:38.861263 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:34:38.861263 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:34:38.861263 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:34:38.861263 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:34:38.861263 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:34:38.861263 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:34:38.861263 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application with Lock( Jul 27 10:34:38.861263 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:34:38.861263 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:34:38.861263 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:34:38.861263 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:34:38.862054 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:34:38.862054 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:34:38.862054 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:34:38.862054 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:34:38.862054 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 10:34:38.862054 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 10:34:38.862054 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 10:34:38.862054 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 10:34:38.862054 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 10:34:38.862054 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 10:34:38.862054 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 27 10:34:38.862054 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Jul 27 10:34:38.862693 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6109daa-d49f-4604-aca2-661bee9b7e8e None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:38.862693 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6109daa-d49f-4604-aca2-661bee9b7e8e None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:38.862693 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6109daa-d49f-4604-aca2-661bee9b7e8e None admin] PATH_INFO: `/v3/groups/56c1bbce28be4b57a92c54fae3188db0` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:38.862879 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 184/379] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jul 27 10:34:38 2022] GET /identity/v3/roles/member => generated 84 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 10:34:38.862879 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f6b902-cee6-4ad0-b1c3-ae5ed1aef641 None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:38.864484 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6109daa-d49f-4604-aca2-661bee9b7e8e None admin] RBAC: Authorizing `identity:get_group(group_id=56c1bbce28be4b57a92c54fae3188db0)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:38.865926 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6109daa-d49f-4604-aca2-661bee9b7e8e None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:38.868555 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 196/380] 10.222.0.36 () {56 vars in 1180 bytes} [Wed Jul 27 10:34:38 2022] GET /identity/v3/groups/56c1bbce28be4b57a92c54fae3188db0 => generated 226 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:34:38.873271 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6109daa-d49f-4604-aca2-661bee9b7e8e None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:38.937406 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:39.001767 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f6b902-cee6-4ad0-b1c3-ae5ed1aef641 None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:39.005990 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6109daa-d49f-4604-aca2-661bee9b7e8e None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:39.039966 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b1d7c5-f646-40bf-a0ef-61d108447569 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:39.040616 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b1d7c5-f646-40bf-a0ef-61d108447569 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:39.040902 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b1d7c5-f646-40bf-a0ef-61d108447569 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:39.041177 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b1d7c5-f646-40bf-a0ef-61d108447569 None admin] PATH_INFO: `/v3/groups/0926913586d841ea9dd4fdd2c32fcb7e` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:39.045102 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b1dcda-ca0b-43fe-af5c-85a035a9301e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:39.045727 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b1dcda-ca0b-43fe-af5c-85a035a9301e None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:39.046013 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b1dcda-ca0b-43fe-af5c-85a035a9301e None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:39.046286 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b1dcda-ca0b-43fe-af5c-85a035a9301e None admin] PATH_INFO: `/v3/roles` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:39.046631 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b1d7c5-f646-40bf-a0ef-61d108447569 None admin] RBAC: Authorizing `identity:get_group(group_id=0926913586d841ea9dd4fdd2c32fcb7e)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:39.047046 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b1dcda-ca0b-43fe-af5c-85a035a9301e None admin] RBAC: Adding query filter params (name=member) {{(pid=86827) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:34:39.047415 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b1dcda-ca0b-43fe-af5c-85a035a9301e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:39.048114 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b1d7c5-f646-40bf-a0ef-61d108447569 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:39.048871 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b1dcda-ca0b-43fe-af5c-85a035a9301e None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:39.050068 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 185/381] 10.222.0.36 () {56 vars in 1180 bytes} [Wed Jul 27 10:34:38 2022] GET /identity/v3/groups/0926913586d841ea9dd4fdd2c32fcb7e => generated 229 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:34:39.050885 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b1d7c5-f646-40bf-a0ef-61d108447569 None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:39.067759 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 197/382] 10.222.0.36 () {56 vars in 1101 bytes} [Wed Jul 27 10:34:38 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 10:34:39.069513 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b1dcda-ca0b-43fe-af5c-85a035a9301e None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:39.093813 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:39.162644 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b1d7c5-f646-40bf-a0ef-61d108447569 None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:39.180495 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:39.183622 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b1dcda-ca0b-43fe-af5c-85a035a9301e None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:39.202450 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2353ebdb-a55d-4c9d-9a16-e56d06764947 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:39.203131 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2353ebdb-a55d-4c9d-9a16-e56d06764947 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:39.203413 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2353ebdb-a55d-4c9d-9a16-e56d06764947 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:39.203684 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2353ebdb-a55d-4c9d-9a16-e56d06764947 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:39.204490 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2353ebdb-a55d-4c9d-9a16-e56d06764947 None admin] RBAC: Adding query filter params (group.id=56c1bbce28be4b57a92c54fae3188db0, role.id=c3341f95596344309327efff99a20d5c, scope.project.id=046cff0c48ee4df282871e4eda9cf939) {{(pid=86828) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:34:39.204801 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2353ebdb-a55d-4c9d-9a16-e56d06764947 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:39.206354 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2353ebdb-a55d-4c9d-9a16-e56d06764947 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:39.213173 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 186/383] 10.222.0.36 () {56 vars in 1376 bytes} [Wed Jul 27 10:34:39 2022] GET /identity/v3/role_assignments?group.id=56c1bbce28be4b57a92c54fae3188db0&scope.project.id=046cff0c48ee4df282871e4eda9cf939&role.id=c3341f95596344309327efff99a20d5c => generated 260 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:34:39.214053 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2353ebdb-a55d-4c9d-9a16-e56d06764947 None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:39.221652 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc027cb5-bfcb-4647-8ebf-52657f7fd803 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:39.222320 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc027cb5-bfcb-4647-8ebf-52657f7fd803 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:39.222623 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc027cb5-bfcb-4647-8ebf-52657f7fd803 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:39.223041 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc027cb5-bfcb-4647-8ebf-52657f7fd803 None admin] PATH_INFO: `/v3/groups/56c1bbce28be4b57a92c54fae3188db0` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:39.224327 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc027cb5-bfcb-4647-8ebf-52657f7fd803 None admin] RBAC: Authorizing `identity:get_group(group_id=56c1bbce28be4b57a92c54fae3188db0)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:39.225840 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc027cb5-bfcb-4647-8ebf-52657f7fd803 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:39.233315 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 198/384] 10.222.0.36 () {56 vars in 1180 bytes} [Wed Jul 27 10:34:39 2022] GET /identity/v3/groups/56c1bbce28be4b57a92c54fae3188db0 => generated 226 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:34:39.234787 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc027cb5-bfcb-4647-8ebf-52657f7fd803 None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:39.330579 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2353ebdb-a55d-4c9d-9a16-e56d06764947 None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:39.336217 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:39.352615 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc027cb5-bfcb-4647-8ebf-52657f7fd803 None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:39.370958 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a9cbc3-8fb3-4bb1-bddc-b0ff64ae419d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:39.371700 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a9cbc3-8fb3-4bb1-bddc-b0ff64ae419d None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:39.372002 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a9cbc3-8fb3-4bb1-bddc-b0ff64ae419d None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:39.372388 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a9cbc3-8fb3-4bb1-bddc-b0ff64ae419d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:39.373211 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a9cbc3-8fb3-4bb1-bddc-b0ff64ae419d None admin] RBAC: Adding query filter params (group.id=0926913586d841ea9dd4fdd2c32fcb7e, role.id=21156560eb6b436bb567a40b4976f3f4, scope.project.id=eeee704de4024dd1b602d1a289cba407) {{(pid=86828) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:34:39.373523 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a9cbc3-8fb3-4bb1-bddc-b0ff64ae419d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:39.375129 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a9cbc3-8fb3-4bb1-bddc-b0ff64ae419d None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:39.382086 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 187/385] 10.222.0.36 () {56 vars in 1376 bytes} [Wed Jul 27 10:34:39 2022] GET /identity/v3/role_assignments?group.id=0926913586d841ea9dd4fdd2c32fcb7e&scope.project.id=eeee704de4024dd1b602d1a289cba407&role.id=21156560eb6b436bb567a40b4976f3f4 => generated 260 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:34:39.399086 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8f37c6-26fc-4e0f-a84c-9d970e250af5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:39.399833 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8f37c6-26fc-4e0f-a84c-9d970e250af5 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:39.400122 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8f37c6-26fc-4e0f-a84c-9d970e250af5 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:39.400407 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8f37c6-26fc-4e0f-a84c-9d970e250af5 None admin] PATH_INFO: `/v3/projects/33cf2502c8694374bd447d6304e658c2` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:39.401811 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8f37c6-26fc-4e0f-a84c-9d970e250af5 None admin] RBAC: Authorizing `identity:get_project(project_id=33cf2502c8694374bd447d6304e658c2)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:39.403467 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8f37c6-26fc-4e0f-a84c-9d970e250af5 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:39.405788 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 199/386] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Jul 27 10:34:39 2022] GET /identity/v3/projects/33cf2502c8694374bd447d6304e658c2 => generated 298 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:34:39.411668 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a9cbc3-8fb3-4bb1-bddc-b0ff64ae419d None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:39.467051 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:39.537064 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a9cbc3-8fb3-4bb1-bddc-b0ff64ae419d None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:39.576297 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842d26de-e127-4f60-9f42-ba02c63f2c7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:39.576993 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842d26de-e127-4f60-9f42-ba02c63f2c7e None admin] REQUEST_METHOD: `PUT` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:39.577278 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842d26de-e127-4f60-9f42-ba02c63f2c7e None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:39.577554 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842d26de-e127-4f60-9f42-ba02c63f2c7e None admin] PATH_INFO: `/v3/projects/33cf2502c8694374bd447d6304e658c2/groups/56c1bbce28be4b57a92c54fae3188db0/roles/4c9fc0b3baed441da2126be41cc6b04a` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:39.579596 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842d26de-e127-4f60-9f42-ba02c63f2c7e None admin] RBAC: Authorizing `identity:create_grant(project_id=33cf2502c8694374bd447d6304e658c2, group_id=56c1bbce28be4b57a92c54fae3188db0, role_id=4c9fc0b3baed441da2126be41cc6b04a)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:39.581113 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842d26de-e127-4f60-9f42-ba02c63f2c7e None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:39.597142 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 188/387] 10.222.0.36 () {58 vars in 1442 bytes} [Wed Jul 27 10:34:39 2022] PUT /identity/v3/projects/33cf2502c8694374bd447d6304e658c2/groups/56c1bbce28be4b57a92c54fae3188db0/roles/4c9fc0b3baed441da2126be41cc6b04a => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:34:39.952716 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2a9443-4413-4abd-b344-51533f752564 None None] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:39.953053 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2a9443-4413-4abd-b344-51533f752564 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:39.953273 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2a9443-4413-4abd-b344-51533f752564 None None] PATH_INFO: `/` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:39.954973 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 200/388] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 10:34:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 10:34:39.961579 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcf03a0-d710-4df1-b12a-c9ded71c4396 None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:39.961874 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcf03a0-d710-4df1-b12a-c9ded71c4396 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:39.962138 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcf03a0-d710-4df1-b12a-c9ded71c4396 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:39.990450 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-8bcf03a0-d710-4df1-b12a-c9ded71c4396 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:34:39.991835 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-8bcf03a0-d710-4df1-b12a-c9ded71c4396 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:34:40.089527 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 189/389] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 10:34:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 27 10:34:40.119940 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2a9443-4413-4abd-b344-51533f752564 None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:40.258447 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2a9443-4413-4abd-b344-51533f752564 None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:40.301711 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89aebd4a-cbde-4a0c-8718-b7474082b30c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:40.302782 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89aebd4a-cbde-4a0c-8718-b7474082b30c None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:40.303265 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89aebd4a-cbde-4a0c-8718-b7474082b30c None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:40.303737 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89aebd4a-cbde-4a0c-8718-b7474082b30c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:40.327849 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89aebd4a-cbde-4a0c-8718-b7474082b30c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 10:34:40.337662 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application [None req-89aebd4a-cbde-4a0c-8718-b7474082b30c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 27 10:34:40.337662 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:34:40.337662 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:34:40.337662 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:34:40.337662 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:34:40.337662 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:34:40.337662 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:34:40.337662 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:34:40.337662 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:34:40.337662 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:34:40.337662 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:34:40.337662 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:34:40.337662 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 10:34:40.337662 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 10:34:40.337662 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 10:34:40.337662 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 10:34:40.338615 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 10:34:40.338615 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 10:34:40.338615 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 10:34:40.338615 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 10:34:40.338615 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:34:40.338615 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:34:40.338615 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:34:40.338615 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:34:40.338615 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:34:40.338615 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:34:40.338615 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:34:40.338615 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application with Lock( Jul 27 10:34:40.338615 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:34:40.338615 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:34:40.338615 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:34:40.338615 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:34:40.339446 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:34:40.339446 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:34:40.339446 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:34:40.339446 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:34:40.339446 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 10:34:40.339446 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 10:34:40.339446 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 10:34:40.339446 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 10:34:40.339446 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 10:34:40.339446 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 10:34:40.339446 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 27 10:34:40.339446 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Jul 27 10:34:40.342171 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 201/390] 10.222.0.36 () {56 vars in 1114 bytes} [Wed Jul 27 10:34:40 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 10:34:40.347729 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcf03a0-d710-4df1-b12a-c9ded71c4396 None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:40.401840 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:40.476240 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcf03a0-d710-4df1-b12a-c9ded71c4396 None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:40.514128 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2421e92c-a331-4d9b-9776-1422f1ebc57e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:40.514700 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2421e92c-a331-4d9b-9776-1422f1ebc57e None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:40.514982 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2421e92c-a331-4d9b-9776-1422f1ebc57e None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:40.515253 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2421e92c-a331-4d9b-9776-1422f1ebc57e None admin] PATH_INFO: `/v3/roles` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:40.515972 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2421e92c-a331-4d9b-9776-1422f1ebc57e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86828) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:34:40.516273 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2421e92c-a331-4d9b-9776-1422f1ebc57e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:40.517633 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2421e92c-a331-4d9b-9776-1422f1ebc57e None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:40.526363 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 190/391] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Jul 27 10:34:40 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:34:40.532047 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89aebd4a-cbde-4a0c-8718-b7474082b30c None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:40.556860 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:40.660395 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89aebd4a-cbde-4a0c-8718-b7474082b30c None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:40.701645 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5453ae7e-98a9-4e12-b362-25999d216e35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:40.702317 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5453ae7e-98a9-4e12-b362-25999d216e35 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:40.702587 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5453ae7e-98a9-4e12-b362-25999d216e35 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:40.702892 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5453ae7e-98a9-4e12-b362-25999d216e35 None admin] PATH_INFO: `/v3/groups/56c1bbce28be4b57a92c54fae3188db0` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:40.704103 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5453ae7e-98a9-4e12-b362-25999d216e35 None admin] RBAC: Authorizing `identity:get_group(group_id=56c1bbce28be4b57a92c54fae3188db0)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:40.705570 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5453ae7e-98a9-4e12-b362-25999d216e35 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:40.707208 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 202/392] 10.222.0.36 () {56 vars in 1180 bytes} [Wed Jul 27 10:34:40 2022] GET /identity/v3/groups/56c1bbce28be4b57a92c54fae3188db0 => generated 226 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:34:40.713409 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2421e92c-a331-4d9b-9776-1422f1ebc57e None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:40.814227 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:40.822061 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2421e92c-a331-4d9b-9776-1422f1ebc57e None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:40.856523 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6708a230-3209-4e81-89ed-bbc431e9b3aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:40.857581 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6708a230-3209-4e81-89ed-bbc431e9b3aa None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:40.858076 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6708a230-3209-4e81-89ed-bbc431e9b3aa None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:40.858561 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6708a230-3209-4e81-89ed-bbc431e9b3aa None admin] PATH_INFO: `/v3/projects/33cf2502c8694374bd447d6304e658c2` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:40.860274 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6708a230-3209-4e81-89ed-bbc431e9b3aa None admin] RBAC: Authorizing `identity:get_project(project_id=33cf2502c8694374bd447d6304e658c2)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:40.862172 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6708a230-3209-4e81-89ed-bbc431e9b3aa None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:40.864807 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 191/393] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Jul 27 10:34:40 2022] GET /identity/v3/projects/33cf2502c8694374bd447d6304e658c2 => generated 298 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:34:40.871220 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5453ae7e-98a9-4e12-b362-25999d216e35 None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:40.981603 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:41.004267 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5453ae7e-98a9-4e12-b362-25999d216e35 None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:41.053653 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906a762f-8448-46ba-bb7f-ca191cab8e8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:41.054873 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906a762f-8448-46ba-bb7f-ca191cab8e8b None admin] REQUEST_METHOD: `PUT` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:41.055161 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906a762f-8448-46ba-bb7f-ca191cab8e8b None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:41.055438 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906a762f-8448-46ba-bb7f-ca191cab8e8b None admin] PATH_INFO: `/v3/projects/33cf2502c8694374bd447d6304e658c2/groups/56c1bbce28be4b57a92c54fae3188db0/roles/c3341f95596344309327efff99a20d5c` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:41.060376 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906a762f-8448-46ba-bb7f-ca191cab8e8b None admin] RBAC: Authorizing `identity:create_grant(project_id=33cf2502c8694374bd447d6304e658c2, group_id=56c1bbce28be4b57a92c54fae3188db0, role_id=c3341f95596344309327efff99a20d5c)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:41.061840 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906a762f-8448-46ba-bb7f-ca191cab8e8b None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:41.079160 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2805c3bd-453a-4fb9-b7fe-f34f9e30cb64 None None] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:41.079448 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2805c3bd-453a-4fb9-b7fe-f34f9e30cb64 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:41.079661 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2805c3bd-453a-4fb9-b7fe-f34f9e30cb64 None None] PATH_INFO: `/` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:41.079764 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 203/394] 10.222.0.36 () {58 vars in 1442 bytes} [Wed Jul 27 10:34:40 2022] PUT /identity/v3/projects/33cf2502c8694374bd447d6304e658c2/groups/56c1bbce28be4b57a92c54fae3188db0/roles/c3341f95596344309327efff99a20d5c => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:34:41.081003 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 192/395] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 10:34:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 10:34:41.098919 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295bb1b5-a17d-4f79-8623-c58f7ec8ee1b None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:41.099288 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295bb1b5-a17d-4f79-8623-c58f7ec8ee1b None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:41.099487 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295bb1b5-a17d-4f79-8623-c58f7ec8ee1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:41.129877 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-295bb1b5-a17d-4f79-8623-c58f7ec8ee1b None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:34:41.131153 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-295bb1b5-a17d-4f79-8623-c58f7ec8ee1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:34:41.230965 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 204/396] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 10:34:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 27 10:34:41.255242 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2805c3bd-453a-4fb9-b7fe-f34f9e30cb64 None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:41.384964 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2805c3bd-453a-4fb9-b7fe-f34f9e30cb64 None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:41.426242 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c28ad4f-5b7d-4c86-aa31-aabe25f9d18d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:41.426930 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c28ad4f-5b7d-4c86-aa31-aabe25f9d18d None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:41.427216 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c28ad4f-5b7d-4c86-aa31-aabe25f9d18d None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:41.427497 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c28ad4f-5b7d-4c86-aa31-aabe25f9d18d None admin] PATH_INFO: `/v3/roles/member` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:41.446337 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c28ad4f-5b7d-4c86-aa31-aabe25f9d18d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 10:34:41.452521 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application [None req-2c28ad4f-5b7d-4c86-aa31-aabe25f9d18d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 27 10:34:41.452521 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:34:41.452521 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:34:41.452521 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:34:41.452521 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:34:41.452521 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:34:41.452521 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:34:41.452521 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:34:41.452521 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:34:41.452521 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:34:41.452521 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:34:41.452521 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:34:41.452521 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 10:34:41.452521 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 10:34:41.452521 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 10:34:41.452521 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 10:34:41.453488 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 10:34:41.453488 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 10:34:41.453488 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 10:34:41.453488 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 10:34:41.453488 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:34:41.453488 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:34:41.453488 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:34:41.453488 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:34:41.453488 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:34:41.453488 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:34:41.453488 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:34:41.453488 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application with Lock( Jul 27 10:34:41.453488 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:34:41.453488 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:34:41.453488 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:34:41.453488 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:34:41.454242 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:34:41.454242 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:34:41.454242 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:34:41.454242 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:34:41.454242 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 10:34:41.454242 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 10:34:41.454242 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 10:34:41.454242 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 10:34:41.454242 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 10:34:41.454242 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 10:34:41.454242 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 27 10:34:41.454242 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Jul 27 10:34:41.454242 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 193/397] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jul 27 10:34:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 10:34:41.463725 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295bb1b5-a17d-4f79-8623-c58f7ec8ee1b None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:41.577613 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:41.614964 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295bb1b5-a17d-4f79-8623-c58f7ec8ee1b None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:41.672740 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7ab5cc-6ac4-4af9-b47d-43a8394134f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:41.673724 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7ab5cc-6ac4-4af9-b47d-43a8394134f6 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:41.674220 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7ab5cc-6ac4-4af9-b47d-43a8394134f6 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:41.674733 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7ab5cc-6ac4-4af9-b47d-43a8394134f6 None admin] PATH_INFO: `/v3/roles` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:41.675661 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7ab5cc-6ac4-4af9-b47d-43a8394134f6 None admin] RBAC: Adding query filter params (name=member) {{(pid=86827) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:34:41.676167 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7ab5cc-6ac4-4af9-b47d-43a8394134f6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:41.677736 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7ab5cc-6ac4-4af9-b47d-43a8394134f6 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:41.690087 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 205/398] 10.222.0.36 () {56 vars in 1101 bytes} [Wed Jul 27 10:34:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:34:41.696329 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c28ad4f-5b7d-4c86-aa31-aabe25f9d18d None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:41.752473 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:41.863989 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2b8e55-c027-4ffa-a4e3-1f1c66970416 None None] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:41.864334 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2b8e55-c027-4ffa-a4e3-1f1c66970416 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:41.864619 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2b8e55-c027-4ffa-a4e3-1f1c66970416 None None] PATH_INFO: `/` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:41.866314 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 206/399] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 10:34:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 10:34:41.871923 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebaef6b-8a36-4ff0-bc79-476803f753ae None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:41.872221 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebaef6b-8a36-4ff0-bc79-476803f753ae None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:41.872507 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebaef6b-8a36-4ff0-bc79-476803f753ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:41.892316 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-0ebaef6b-8a36-4ff0-bc79-476803f753ae None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:34:41.893537 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-0ebaef6b-8a36-4ff0-bc79-476803f753ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:34:41.910492 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c28ad4f-5b7d-4c86-aa31-aabe25f9d18d None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:41.976667 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e461e67-5611-41fa-896a-dffdf00effc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:41.977324 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e461e67-5611-41fa-896a-dffdf00effc9 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:41.977608 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e461e67-5611-41fa-896a-dffdf00effc9 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:41.977939 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e461e67-5611-41fa-896a-dffdf00effc9 None admin] PATH_INFO: `/v3/groups/56c1bbce28be4b57a92c54fae3188db0` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:41.980285 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e461e67-5611-41fa-896a-dffdf00effc9 None admin] RBAC: Authorizing `identity:get_group(group_id=56c1bbce28be4b57a92c54fae3188db0)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:41.981736 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e461e67-5611-41fa-896a-dffdf00effc9 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:41.984188 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 194/400] 10.222.0.36 () {56 vars in 1180 bytes} [Wed Jul 27 10:34:41 2022] GET /identity/v3/groups/56c1bbce28be4b57a92c54fae3188db0 => generated 226 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:34:41.992118 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e461e67-5611-41fa-896a-dffdf00effc9 None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:42.005408 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 207/401] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 10:34:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 10:34:42.044406 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c54fd34-acfc-4348-ba91-f0a2236303e9 None None] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:42.045019 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c54fd34-acfc-4348-ba91-f0a2236303e9 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:42.045496 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c54fd34-acfc-4348-ba91-f0a2236303e9 None None] PATH_INFO: `/` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:42.047366 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 208/402] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 10:34:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 10:34:42.052470 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cc5bc4-e1b1-403f-9f11-59d7890d0d8a None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:42.053019 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cc5bc4-e1b1-403f-9f11-59d7890d0d8a None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:42.053509 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cc5bc4-e1b1-403f-9f11-59d7890d0d8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:42.075933 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-31cc5bc4-e1b1-403f-9f11-59d7890d0d8a None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:34:42.078202 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-31cc5bc4-e1b1-403f-9f11-59d7890d0d8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:34:42.113471 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:42.126528 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e461e67-5611-41fa-896a-dffdf00effc9 None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:42.162352 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:42.166151 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1890e025-7f08-4537-a9c2-7c3a199b2fba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:42.167689 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1890e025-7f08-4537-a9c2-7c3a199b2fba None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:42.167981 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1890e025-7f08-4537-a9c2-7c3a199b2fba None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:42.168273 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1890e025-7f08-4537-a9c2-7c3a199b2fba None admin] PATH_INFO: `/v3/projects/046cff0c48ee4df282871e4eda9cf939` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:42.169467 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1890e025-7f08-4537-a9c2-7c3a199b2fba None admin] RBAC: Authorizing `identity:get_project(project_id=046cff0c48ee4df282871e4eda9cf939)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:42.171215 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1890e025-7f08-4537-a9c2-7c3a199b2fba None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:42.177985 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 195/404] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Jul 27 10:34:41 2022] GET /identity/v3/projects/046cff0c48ee4df282871e4eda9cf939 => generated 302 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:34:42.178284 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 209/404] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 10:34:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 27 10:34:42.178857 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1890e025-7f08-4537-a9c2-7c3a199b2fba None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:42.180295 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cc5bc4-e1b1-403f-9f11-59d7890d0d8a None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:42.299221 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1890e025-7f08-4537-a9c2-7c3a199b2fba None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:42.330253 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cc5bc4-e1b1-403f-9f11-59d7890d0d8a None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:42.333582 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37054226-dbc1-4e3f-9303-2e9fcdfb43e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:42.334231 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37054226-dbc1-4e3f-9303-2e9fcdfb43e4 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:42.334518 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37054226-dbc1-4e3f-9303-2e9fcdfb43e4 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:42.334840 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37054226-dbc1-4e3f-9303-2e9fcdfb43e4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:42.345040 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37054226-dbc1-4e3f-9303-2e9fcdfb43e4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 10:34:42.350446 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application [None req-37054226-dbc1-4e3f-9303-2e9fcdfb43e4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 10:34:42.350446 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:34:42.350446 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:34:42.350446 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:34:42.350446 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:34:42.350446 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:34:42.350446 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:34:42.350446 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:34:42.350446 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:34:42.350446 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:34:42.350446 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:34:42.350446 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:34:42.350446 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 10:34:42.350446 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 10:34:42.350446 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 10:34:42.350446 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 10:34:42.351898 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 10:34:42.351898 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 10:34:42.351898 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 10:34:42.351898 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 10:34:42.351898 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:34:42.351898 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:34:42.351898 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:34:42.351898 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:34:42.351898 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:34:42.351898 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:34:42.351898 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:34:42.351898 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application with Lock( Jul 27 10:34:42.351898 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:34:42.351898 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:34:42.351898 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:34:42.351898 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:34:42.352683 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:34:42.352683 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:34:42.352683 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:34:42.352683 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:34:42.352683 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 10:34:42.352683 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 10:34:42.352683 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 10:34:42.352683 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 10:34:42.352683 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 10:34:42.352683 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 10:34:42.352683 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 10:34:42.352683 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Jul 27 10:34:42.352683 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 196/405] 10.222.0.36 () {56 vars in 1096 bytes} [Wed Jul 27 10:34:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 10:34:42.357739 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37054226-dbc1-4e3f-9303-2e9fcdfb43e4 None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:42.365496 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75df57f-2a20-45e0-9524-62fbc2337896 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:42.366212 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75df57f-2a20-45e0-9524-62fbc2337896 None admin] REQUEST_METHOD: `PUT` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:42.366498 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75df57f-2a20-45e0-9524-62fbc2337896 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:42.367439 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75df57f-2a20-45e0-9524-62fbc2337896 None admin] PATH_INFO: `/v3/projects/046cff0c48ee4df282871e4eda9cf939/groups/56c1bbce28be4b57a92c54fae3188db0/roles/4c9fc0b3baed441da2126be41cc6b04a` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:42.369455 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75df57f-2a20-45e0-9524-62fbc2337896 None admin] RBAC: Authorizing `identity:create_grant(project_id=046cff0c48ee4df282871e4eda9cf939, group_id=56c1bbce28be4b57a92c54fae3188db0, role_id=4c9fc0b3baed441da2126be41cc6b04a)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:42.370902 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75df57f-2a20-45e0-9524-62fbc2337896 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:42.378245 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:42.385685 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 210/406] 10.222.0.36 () {58 vars in 1442 bytes} [Wed Jul 27 10:34:42 2022] PUT /identity/v3/projects/046cff0c48ee4df282871e4eda9cf939/groups/56c1bbce28be4b57a92c54fae3188db0/roles/4c9fc0b3baed441da2126be41cc6b04a => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:34:42.387201 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777ce676-4d5d-4f76-8537-181f80614538 None None] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:42.387554 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777ce676-4d5d-4f76-8537-181f80614538 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:42.387900 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777ce676-4d5d-4f76-8537-181f80614538 None None] PATH_INFO: `/` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:42.389432 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 211/407] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 10:34:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 10:34:42.394179 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777ce676-4d5d-4f76-8537-181f80614538 None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:42.464266 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:42.505267 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37054226-dbc1-4e3f-9303-2e9fcdfb43e4 None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:42.517799 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777ce676-4d5d-4f76-8537-181f80614538 None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:42.559011 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c010d4c1-99be-4206-9885-77e6682c5a4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:42.560026 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c010d4c1-99be-4206-9885-77e6682c5a4a None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:42.560632 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c010d4c1-99be-4206-9885-77e6682c5a4a None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:42.561186 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c010d4c1-99be-4206-9885-77e6682c5a4a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:42.562059 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91eb2515-e9bc-4887-be83-79812c711fff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:42.562837 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91eb2515-e9bc-4887-be83-79812c711fff None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:42.563122 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91eb2515-e9bc-4887-be83-79812c711fff None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:42.563697 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91eb2515-e9bc-4887-be83-79812c711fff None admin] PATH_INFO: `/v3/roles` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:42.564901 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91eb2515-e9bc-4887-be83-79812c711fff None admin] RBAC: Adding query filter params (name=admin) {{(pid=86827) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:34:42.565254 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91eb2515-e9bc-4887-be83-79812c711fff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:42.566764 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91eb2515-e9bc-4887-be83-79812c711fff None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:42.574580 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c010d4c1-99be-4206-9885-77e6682c5a4a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 10:34:42.577596 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 212/408] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jul 27 10:34:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:34:42.578862 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9838723d-7dba-4a99-b641-a1c800b74fe0 None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:42.579159 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9838723d-7dba-4a99-b641-a1c800b74fe0 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:42.579443 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9838723d-7dba-4a99-b641-a1c800b74fe0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:42.581435 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application [None req-c010d4c1-99be-4206-9885-77e6682c5a4a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 27 10:34:42.581435 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:34:42.581435 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:34:42.581435 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:34:42.581435 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:34:42.581435 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:34:42.581435 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:34:42.581435 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:34:42.581435 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:34:42.581435 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:34:42.581435 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:34:42.581435 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:34:42.581435 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 10:34:42.581435 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 10:34:42.581435 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 10:34:42.581435 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 10:34:42.582388 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 10:34:42.582388 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 10:34:42.582388 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 10:34:42.582388 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 10:34:42.582388 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:34:42.582388 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:34:42.582388 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:34:42.582388 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:34:42.582388 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:34:42.582388 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:34:42.582388 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:34:42.582388 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application with Lock( Jul 27 10:34:42.582388 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:34:42.582388 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:34:42.582388 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:34:42.582388 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:34:42.583262 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:34:42.583262 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:34:42.583262 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:34:42.583262 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:34:42.583262 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 10:34:42.583262 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 10:34:42.583262 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 10:34:42.583262 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 10:34:42.583262 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 10:34:42.583262 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 10:34:42.583262 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 27 10:34:42.583262 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Jul 27 10:34:42.583262 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 197/409] 10.222.0.36 () {56 vars in 1114 bytes} [Wed Jul 27 10:34:42 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 10:34:42.590079 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c010d4c1-99be-4206-9885-77e6682c5a4a None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:42.616179 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-9838723d-7dba-4a99-b641-a1c800b74fe0 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:34:42.618228 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-9838723d-7dba-4a99-b641-a1c800b74fe0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:34:42.703619 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:42.711050 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 213/410] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 10:34:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 10:34:42.712241 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9838723d-7dba-4a99-b641-a1c800b74fe0 None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:42.729131 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c010d4c1-99be-4206-9885-77e6682c5a4a None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:42.773799 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:42.775891 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b382496-8cda-4929-90e6-3d5cd1afa4ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:42.776974 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b382496-8cda-4929-90e6-3d5cd1afa4ab None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:42.777374 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b382496-8cda-4929-90e6-3d5cd1afa4ab None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:42.777722 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b382496-8cda-4929-90e6-3d5cd1afa4ab None admin] PATH_INFO: `/v3/groups/0926913586d841ea9dd4fdd2c32fcb7e` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:42.779083 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b382496-8cda-4929-90e6-3d5cd1afa4ab None admin] RBAC: Authorizing `identity:get_group(group_id=0926913586d841ea9dd4fdd2c32fcb7e)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:42.780605 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b382496-8cda-4929-90e6-3d5cd1afa4ab None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:42.783406 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 198/411] 10.222.0.36 () {56 vars in 1180 bytes} [Wed Jul 27 10:34:42 2022] GET /identity/v3/groups/0926913586d841ea9dd4fdd2c32fcb7e => generated 229 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 10:34:42.788016 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b382496-8cda-4929-90e6-3d5cd1afa4ab None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:42.853578 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9838723d-7dba-4a99-b641-a1c800b74fe0 None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:42.896795 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b382496-8cda-4929-90e6-3d5cd1afa4ab None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:42.900281 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54857352-2602-4d30-ad82-8de7c38b902f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:42.900919 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54857352-2602-4d30-ad82-8de7c38b902f None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:42.901206 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54857352-2602-4d30-ad82-8de7c38b902f None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:42.901482 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54857352-2602-4d30-ad82-8de7c38b902f None admin] PATH_INFO: `/v3/roles` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:42.902216 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54857352-2602-4d30-ad82-8de7c38b902f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86827) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:34:42.902524 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54857352-2602-4d30-ad82-8de7c38b902f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:42.903935 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54857352-2602-4d30-ad82-8de7c38b902f None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:42.912729 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 214/412] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Jul 27 10:34:42 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:34:42.913683 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54857352-2602-4d30-ad82-8de7c38b902f None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:42.941981 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f32354-e373-4d38-a7e4-685c05ce11c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:42.943041 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f32354-e373-4d38-a7e4-685c05ce11c2 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:42.943573 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f32354-e373-4d38-a7e4-685c05ce11c2 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:42.944054 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f32354-e373-4d38-a7e4-685c05ce11c2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:42.965132 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f32354-e373-4d38-a7e4-685c05ce11c2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 10:34:42.976890 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application [None req-b4f32354-e373-4d38-a7e4-685c05ce11c2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 27 10:34:42.976890 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:34:42.976890 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:34:42.976890 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:34:42.976890 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:34:42.976890 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:34:42.976890 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:34:42.976890 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:34:42.976890 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:34:42.976890 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:34:42.976890 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:34:42.976890 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:34:42.976890 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 10:34:42.976890 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 10:34:42.976890 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 10:34:42.976890 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 10:34:42.978483 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 10:34:42.978483 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 10:34:42.978483 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 10:34:42.978483 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 10:34:42.978483 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:34:42.978483 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:34:42.978483 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:34:42.978483 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:34:42.978483 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:34:42.978483 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:34:42.978483 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:34:42.978483 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application with Lock( Jul 27 10:34:42.978483 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:34:42.978483 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:34:42.978483 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:34:42.978483 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:34:42.979997 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:34:42.979997 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:34:42.979997 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:34:42.979997 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:34:42.979997 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 10:34:42.979997 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 10:34:42.979997 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 10:34:42.979997 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 10:34:42.979997 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 10:34:42.979997 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 10:34:42.979997 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 27 10:34:42.979997 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Jul 27 10:34:42.983697 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 199/413] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jul 27 10:34:42 2022] GET /identity/v3/roles/member => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 10:34:42.985058 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f32354-e373-4d38-a7e4-685c05ce11c2 None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:42.996451 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:43.030016 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54857352-2602-4d30-ad82-8de7c38b902f None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:43.041916 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:43.069035 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7736c1-403e-44c5-bf5b-9c1e40e28f27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:43.070063 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7736c1-403e-44c5-bf5b-9c1e40e28f27 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:43.070493 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7736c1-403e-44c5-bf5b-9c1e40e28f27 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:43.070961 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7736c1-403e-44c5-bf5b-9c1e40e28f27 None admin] PATH_INFO: `/v3/projects/eeee704de4024dd1b602d1a289cba407` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:43.072313 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7736c1-403e-44c5-bf5b-9c1e40e28f27 None admin] RBAC: Authorizing `identity:get_project(project_id=eeee704de4024dd1b602d1a289cba407)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:43.074101 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7736c1-403e-44c5-bf5b-9c1e40e28f27 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:43.076468 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 215/414] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Jul 27 10:34:42 2022] GET /identity/v3/projects/eeee704de4024dd1b602d1a289cba407 => generated 344 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:34:43.077392 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7736c1-403e-44c5-bf5b-9c1e40e28f27 None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:43.133198 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f32354-e373-4d38-a7e4-685c05ce11c2 None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:43.171127 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b183185e-55cf-42ed-a533-83fe5c1490c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:43.172015 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b183185e-55cf-42ed-a533-83fe5c1490c1 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:43.172397 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b183185e-55cf-42ed-a533-83fe5c1490c1 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:43.172771 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b183185e-55cf-42ed-a533-83fe5c1490c1 None admin] PATH_INFO: `/v3/groups/56c1bbce28be4b57a92c54fae3188db0` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:43.174139 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b183185e-55cf-42ed-a533-83fe5c1490c1 None admin] RBAC: Authorizing `identity:get_group(group_id=56c1bbce28be4b57a92c54fae3188db0)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:43.175733 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b183185e-55cf-42ed-a533-83fe5c1490c1 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:43.177565 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 200/415] 10.222.0.36 () {56 vars in 1180 bytes} [Wed Jul 27 10:34:42 2022] GET /identity/v3/groups/56c1bbce28be4b57a92c54fae3188db0 => generated 226 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:34:43.178390 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b183185e-55cf-42ed-a533-83fe5c1490c1 None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:43.189732 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7736c1-403e-44c5-bf5b-9c1e40e28f27 None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:43.227965 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c4cacd-b10e-4b4a-9437-d045af90b398 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:43.228870 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c4cacd-b10e-4b4a-9437-d045af90b398 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:43.229415 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c4cacd-b10e-4b4a-9437-d045af90b398 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:43.229844 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c4cacd-b10e-4b4a-9437-d045af90b398 None admin] PATH_INFO: `/v3/roles` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:43.230877 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c4cacd-b10e-4b4a-9437-d045af90b398 None admin] RBAC: Adding query filter params (name=member) {{(pid=86827) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:34:43.231331 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c4cacd-b10e-4b4a-9437-d045af90b398 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:43.232854 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c4cacd-b10e-4b4a-9437-d045af90b398 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:43.242386 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 216/416] 10.222.0.36 () {56 vars in 1101 bytes} [Wed Jul 27 10:34:43 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:34:43.243545 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c4cacd-b10e-4b4a-9437-d045af90b398 None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:43.262391 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:43.293931 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:43.305424 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b183185e-55cf-42ed-a533-83fe5c1490c1 None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:43.353649 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0b3940-c1db-4a71-9016-0ae9cb48be8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:43.354324 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0b3940-c1db-4a71-9016-0ae9cb48be8f None admin] REQUEST_METHOD: `PUT` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:43.354625 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0b3940-c1db-4a71-9016-0ae9cb48be8f None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:43.354934 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0b3940-c1db-4a71-9016-0ae9cb48be8f None admin] PATH_INFO: `/v3/projects/eeee704de4024dd1b602d1a289cba407/groups/0926913586d841ea9dd4fdd2c32fcb7e/roles/21156560eb6b436bb567a40b4976f3f4` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:43.358101 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0b3940-c1db-4a71-9016-0ae9cb48be8f None admin] RBAC: Authorizing `identity:create_grant(project_id=eeee704de4024dd1b602d1a289cba407, group_id=0926913586d841ea9dd4fdd2c32fcb7e, role_id=21156560eb6b436bb567a40b4976f3f4)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:43.359703 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0b3940-c1db-4a71-9016-0ae9cb48be8f None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:43.373442 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c4cacd-b10e-4b4a-9437-d045af90b398 None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:43.378674 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 201/417] 10.222.0.36 () {58 vars in 1442 bytes} [Wed Jul 27 10:34:43 2022] PUT /identity/v3/projects/eeee704de4024dd1b602d1a289cba407/groups/0926913586d841ea9dd4fdd2c32fcb7e/roles/21156560eb6b436bb567a40b4976f3f4 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:34:43.379990 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0b3940-c1db-4a71-9016-0ae9cb48be8f None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:43.448424 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81df786b-b815-4e5c-8471-c9ab0745eac8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:43.449018 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81df786b-b815-4e5c-8471-c9ab0745eac8 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:43.449313 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81df786b-b815-4e5c-8471-c9ab0745eac8 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:43.449589 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81df786b-b815-4e5c-8471-c9ab0745eac8 None admin] PATH_INFO: `/v3/projects/046cff0c48ee4df282871e4eda9cf939` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:43.455512 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81df786b-b815-4e5c-8471-c9ab0745eac8 None admin] RBAC: Authorizing `identity:get_project(project_id=046cff0c48ee4df282871e4eda9cf939)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:43.456995 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81df786b-b815-4e5c-8471-c9ab0745eac8 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:43.458804 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 217/418] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Jul 27 10:34:43 2022] GET /identity/v3/projects/046cff0c48ee4df282871e4eda9cf939 => generated 302 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:34:43.464779 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81df786b-b815-4e5c-8471-c9ab0745eac8 None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:43.533958 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0b3940-c1db-4a71-9016-0ae9cb48be8f None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:43.576538 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b763045b-a409-4fc9-b653-6fa66ca33151 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:43.577171 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b763045b-a409-4fc9-b653-6fa66ca33151 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:43.577457 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b763045b-a409-4fc9-b653-6fa66ca33151 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:43.577735 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b763045b-a409-4fc9-b653-6fa66ca33151 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:43.581383 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:43.651957 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81df786b-b815-4e5c-8471-c9ab0745eac8 None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:43.665905 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:43.675091 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b763045b-a409-4fc9-b653-6fa66ca33151 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:43.677033 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b763045b-a409-4fc9-b653-6fa66ca33151 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:43.699333 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8a5e71-159c-4e3d-b200-3d38ed50ea99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:43.699940 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8a5e71-159c-4e3d-b200-3d38ed50ea99 None admin] REQUEST_METHOD: `PUT` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:43.700227 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8a5e71-159c-4e3d-b200-3d38ed50ea99 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:43.700511 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8a5e71-159c-4e3d-b200-3d38ed50ea99 None admin] PATH_INFO: `/v3/projects/046cff0c48ee4df282871e4eda9cf939/groups/56c1bbce28be4b57a92c54fae3188db0/roles/c3341f95596344309327efff99a20d5c` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:43.703338 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8a5e71-159c-4e3d-b200-3d38ed50ea99 None admin] RBAC: Authorizing `identity:create_grant(project_id=046cff0c48ee4df282871e4eda9cf939, group_id=56c1bbce28be4b57a92c54fae3188db0, role_id=c3341f95596344309327efff99a20d5c)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:43.705071 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8a5e71-159c-4e3d-b200-3d38ed50ea99 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:43.722902 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 218/419] 10.222.0.36 () {58 vars in 1442 bytes} [Wed Jul 27 10:34:43 2022] PUT /identity/v3/projects/046cff0c48ee4df282871e4eda9cf939/groups/56c1bbce28be4b57a92c54fae3188db0/roles/c3341f95596344309327efff99a20d5c => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 10:34:43.724485 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfba0955-216c-42c3-9fd4-309bcf5c50b6 None None] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:43.724988 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfba0955-216c-42c3-9fd4-309bcf5c50b6 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:43.725465 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfba0955-216c-42c3-9fd4-309bcf5c50b6 None None] PATH_INFO: `/` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:43.727597 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 219/420] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 10:34:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 10:34:43.737175 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fe1152-349d-419b-ad41-2b3a7d1e91f3 None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:43.737808 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fe1152-349d-419b-ad41-2b3a7d1e91f3 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:43.738286 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fe1152-349d-419b-ad41-2b3a7d1e91f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:43.779973 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-b8fe1152-349d-419b-ad41-2b3a7d1e91f3 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:34:43.781609 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-b8fe1152-349d-419b-ad41-2b3a7d1e91f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:34:43.812089 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 202/421] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jul 27 10:34:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 10:34:43.821704 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b763045b-a409-4fc9-b653-6fa66ca33151 None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:43.877004 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:43.891771 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 220/422] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 10:34:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 10:34:43.920714 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fe1152-349d-419b-ad41-2b3a7d1e91f3 None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:44.011919 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b763045b-a409-4fc9-b653-6fa66ca33151 None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:44.043491 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:44.049598 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fe1152-349d-419b-ad41-2b3a7d1e91f3 None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:44.057690 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86942c3-f6bf-4f4e-a49d-127c0dcf39ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:44.058792 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86942c3-f6bf-4f4e-a49d-127c0dcf39ce None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:44.059293 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86942c3-f6bf-4f4e-a49d-127c0dcf39ce None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:44.059765 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86942c3-f6bf-4f4e-a49d-127c0dcf39ce None admin] PATH_INFO: `/v3/projects/33cf2502c8694374bd447d6304e658c2` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:44.061153 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86942c3-f6bf-4f4e-a49d-127c0dcf39ce None admin] RBAC: Authorizing `identity:get_project(project_id=33cf2502c8694374bd447d6304e658c2)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:44.062851 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86942c3-f6bf-4f4e-a49d-127c0dcf39ce None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:44.065157 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 203/423] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Jul 27 10:34:43 2022] GET /identity/v3/projects/33cf2502c8694374bd447d6304e658c2 => generated 298 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:34:44.072101 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86942c3-f6bf-4f4e-a49d-127c0dcf39ce None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:44.100421 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf02d5c-15bf-46a3-bd9c-25d6962ea332 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:44.101623 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf02d5c-15bf-46a3-bd9c-25d6962ea332 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:44.101902 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf02d5c-15bf-46a3-bd9c-25d6962ea332 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:44.102171 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf02d5c-15bf-46a3-bd9c-25d6962ea332 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:44.134054 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf02d5c-15bf-46a3-bd9c-25d6962ea332 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 10:34:44.141452 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application [None req-fbf02d5c-15bf-46a3-bd9c-25d6962ea332 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 27 10:34:44.141452 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:34:44.141452 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:34:44.141452 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:34:44.141452 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:34:44.141452 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:34:44.141452 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:34:44.141452 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:34:44.141452 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:34:44.141452 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:34:44.141452 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:34:44.141452 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:34:44.141452 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 10:34:44.141452 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 10:34:44.141452 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 10:34:44.141452 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 10:34:44.142318 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 10:34:44.142318 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 10:34:44.142318 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 10:34:44.142318 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 10:34:44.142318 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:34:44.142318 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:34:44.142318 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:34:44.142318 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:34:44.142318 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:34:44.142318 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:34:44.142318 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:34:44.142318 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application with Lock( Jul 27 10:34:44.142318 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:34:44.142318 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:34:44.142318 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:34:44.142318 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:34:44.143125 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:34:44.143125 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:34:44.143125 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:34:44.143125 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:34:44.143125 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 10:34:44.143125 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 10:34:44.143125 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 10:34:44.143125 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 10:34:44.143125 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 10:34:44.143125 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 10:34:44.143125 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 27 10:34:44.143125 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Jul 27 10:34:44.146971 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 221/424] 10.222.0.36 () {56 vars in 1114 bytes} [Wed Jul 27 10:34:43 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 10:34:44.155038 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf02d5c-15bf-46a3-bd9c-25d6962ea332 None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:44.235924 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86942c3-f6bf-4f4e-a49d-127c0dcf39ce None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:44.238909 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:44.302536 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c0ddaf-8ebf-404e-9479-fbb6774b40b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:44.303571 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c0ddaf-8ebf-404e-9479-fbb6774b40b8 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:44.304080 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c0ddaf-8ebf-404e-9479-fbb6774b40b8 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:44.304551 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c0ddaf-8ebf-404e-9479-fbb6774b40b8 None admin] PATH_INFO: `/v3/groups/56c1bbce28be4b57a92c54fae3188db0` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:44.306874 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c0ddaf-8ebf-404e-9479-fbb6774b40b8 None admin] RBAC: Authorizing `identity:get_group(group_id=56c1bbce28be4b57a92c54fae3188db0)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:44.308543 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c0ddaf-8ebf-404e-9479-fbb6774b40b8 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:44.310485 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 204/425] 10.222.0.36 () {56 vars in 1180 bytes} [Wed Jul 27 10:34:44 2022] GET /identity/v3/groups/56c1bbce28be4b57a92c54fae3188db0 => generated 226 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:34:44.317667 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c0ddaf-8ebf-404e-9479-fbb6774b40b8 None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:44.332163 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf02d5c-15bf-46a3-bd9c-25d6962ea332 None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:44.391283 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ecea3f-2eef-4b26-8397-32bf41550bf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:44.392320 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ecea3f-2eef-4b26-8397-32bf41550bf0 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:44.394787 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ecea3f-2eef-4b26-8397-32bf41550bf0 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:44.399888 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ecea3f-2eef-4b26-8397-32bf41550bf0 None admin] PATH_INFO: `/v3/roles` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:44.401559 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ecea3f-2eef-4b26-8397-32bf41550bf0 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86827) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:34:44.402273 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ecea3f-2eef-4b26-8397-32bf41550bf0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:44.405203 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ecea3f-2eef-4b26-8397-32bf41550bf0 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:44.417119 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 222/426] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Jul 27 10:34:44 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:34:44.423544 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ecea3f-2eef-4b26-8397-32bf41550bf0 None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:44.464192 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:44.494485 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c0ddaf-8ebf-404e-9479-fbb6774b40b8 None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:44.567436 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70204aa9-a409-4063-90a2-02159314ccbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:44.568064 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70204aa9-a409-4063-90a2-02159314ccbb None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:44.568310 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70204aa9-a409-4063-90a2-02159314ccbb None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:44.568537 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70204aa9-a409-4063-90a2-02159314ccbb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:44.569352 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70204aa9-a409-4063-90a2-02159314ccbb None admin] RBAC: Adding query filter params (group.id=56c1bbce28be4b57a92c54fae3188db0, role.id=4c9fc0b3baed441da2126be41cc6b04a, scope.project.id=33cf2502c8694374bd447d6304e658c2) {{(pid=86828) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:34:44.569666 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70204aa9-a409-4063-90a2-02159314ccbb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:44.576135 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70204aa9-a409-4063-90a2-02159314ccbb None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:44.578321 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 205/427] 10.222.0.36 () {56 vars in 1376 bytes} [Wed Jul 27 10:34:44 2022] GET /identity/v3/role_assignments?group.id=56c1bbce28be4b57a92c54fae3188db0&scope.project.id=33cf2502c8694374bd447d6304e658c2&role.id=4c9fc0b3baed441da2126be41cc6b04a => generated 611 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:34:44.593596 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ecea3f-2eef-4b26-8397-32bf41550bf0 None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:44.630079 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81e1894-f34b-49c5-8b38-ce5a4c90d58b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:44.631183 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81e1894-f34b-49c5-8b38-ce5a4c90d58b None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:44.631716 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81e1894-f34b-49c5-8b38-ce5a4c90d58b None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:44.632736 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81e1894-f34b-49c5-8b38-ce5a4c90d58b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:44.660464 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472e8696-a097-4d21-b029-e4f15b54c388 None None] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:44.660772 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472e8696-a097-4d21-b029-e4f15b54c388 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:44.661025 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472e8696-a097-4d21-b029-e4f15b54c388 None None] PATH_INFO: `/` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:44.662636 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 206/428] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 10:34:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 10:34:44.668043 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffa5eea-8062-4dc7-a5d5-fec2b565436e None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:44.669699 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffa5eea-8062-4dc7-a5d5-fec2b565436e None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:44.670374 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffa5eea-8062-4dc7-a5d5-fec2b565436e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:44.706427 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-dffa5eea-8062-4dc7-a5d5-fec2b565436e None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:34:44.711022 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-dffa5eea-8062-4dc7-a5d5-fec2b565436e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:34:44.762219 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:44.764174 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81e1894-f34b-49c5-8b38-ce5a4c90d58b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:44.765904 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81e1894-f34b-49c5-8b38-ce5a4c90d58b None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:44.815456 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:44.825332 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 207/429] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 10:34:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Jul 27 10:34:44.848771 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffa5eea-8062-4dc7-a5d5-fec2b565436e None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:44.896176 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 223/430] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jul 27 10:34:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 473 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 10:34:44.901740 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81e1894-f34b-49c5-8b38-ce5a4c90d58b None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:44.964303 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffa5eea-8062-4dc7-a5d5-fec2b565436e None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:45.008468 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81e1894-f34b-49c5-8b38-ce5a4c90d58b None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:45.019930 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69dcec2d-183b-4d2f-8ad9-212daa627819 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:45.020628 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69dcec2d-183b-4d2f-8ad9-212daa627819 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:45.020912 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69dcec2d-183b-4d2f-8ad9-212daa627819 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:45.021511 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69dcec2d-183b-4d2f-8ad9-212daa627819 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:45.033375 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:45.037062 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69dcec2d-183b-4d2f-8ad9-212daa627819 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 10:34:45.043775 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application [None req-69dcec2d-183b-4d2f-8ad9-212daa627819 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 27 10:34:45.043775 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:34:45.043775 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:34:45.043775 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:34:45.043775 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:34:45.043775 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:34:45.043775 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:34:45.043775 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:34:45.043775 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:34:45.043775 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:34:45.043775 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:34:45.043775 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:34:45.043775 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 10:34:45.043775 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 10:34:45.043775 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 10:34:45.043775 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 10:34:45.044679 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 10:34:45.044679 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 10:34:45.044679 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 10:34:45.044679 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 10:34:45.044679 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:34:45.044679 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:34:45.044679 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:34:45.044679 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:34:45.044679 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:34:45.044679 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:34:45.044679 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:34:45.044679 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application with Lock( Jul 27 10:34:45.044679 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:34:45.044679 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:34:45.044679 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:34:45.044679 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:34:45.045456 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:34:45.045456 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:34:45.045456 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:34:45.045456 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:34:45.045456 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 10:34:45.045456 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 10:34:45.045456 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 10:34:45.045456 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 10:34:45.045456 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 10:34:45.045456 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 10:34:45.045456 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 27 10:34:45.045456 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Jul 27 10:34:45.050544 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 208/431] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jul 27 10:34:44 2022] GET /identity/v3/roles/member => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 10:34:45.051416 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69dcec2d-183b-4d2f-8ad9-212daa627819 None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:45.058744 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3439b3fe-7b58-420b-a536-447548a65371 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:45.059486 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3439b3fe-7b58-420b-a536-447548a65371 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:45.059843 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3439b3fe-7b58-420b-a536-447548a65371 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:45.060181 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3439b3fe-7b58-420b-a536-447548a65371 None admin] PATH_INFO: `/v3/projects/33cf2502c8694374bd447d6304e658c2` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:45.061445 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3439b3fe-7b58-420b-a536-447548a65371 None admin] RBAC: Authorizing `identity:get_project(project_id=33cf2502c8694374bd447d6304e658c2)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:45.063452 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3439b3fe-7b58-420b-a536-447548a65371 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:45.065823 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 224/432] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Jul 27 10:34:44 2022] GET /identity/v3/projects/33cf2502c8694374bd447d6304e658c2 => generated 298 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 10:34:45.072080 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3439b3fe-7b58-420b-a536-447548a65371 None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:45.087096 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:45.178540 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69dcec2d-183b-4d2f-8ad9-212daa627819 None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:45.196299 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3439b3fe-7b58-420b-a536-447548a65371 None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:45.222760 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb4cdad-0dcb-450a-9f50-0a080b79f065 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:45.223680 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb4cdad-0dcb-450a-9f50-0a080b79f065 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:45.224386 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb4cdad-0dcb-450a-9f50-0a080b79f065 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:45.224667 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb4cdad-0dcb-450a-9f50-0a080b79f065 None admin] PATH_INFO: `/v3/roles` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:45.225663 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb4cdad-0dcb-450a-9f50-0a080b79f065 None admin] RBAC: Adding query filter params (name=member) {{(pid=86828) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:34:45.226516 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb4cdad-0dcb-450a-9f50-0a080b79f065 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:45.228072 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb4cdad-0dcb-450a-9f50-0a080b79f065 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:45.237577 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9200925c-633c-40ec-b2a2-122d8aeffb29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:45.238249 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9200925c-633c-40ec-b2a2-122d8aeffb29 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:45.239642 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 209/433] 10.222.0.36 () {56 vars in 1101 bytes} [Wed Jul 27 10:34:45 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:34:45.240134 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9200925c-633c-40ec-b2a2-122d8aeffb29 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:45.240422 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9200925c-633c-40ec-b2a2-122d8aeffb29 None admin] PATH_INFO: `/v3/groups/56c1bbce28be4b57a92c54fae3188db0` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:45.247380 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb4cdad-0dcb-450a-9f50-0a080b79f065 None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:45.247559 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9200925c-633c-40ec-b2a2-122d8aeffb29 None admin] RBAC: Authorizing `identity:get_group(group_id=56c1bbce28be4b57a92c54fae3188db0)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:45.248969 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9200925c-633c-40ec-b2a2-122d8aeffb29 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:45.251056 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 225/434] 10.222.0.36 () {56 vars in 1180 bytes} [Wed Jul 27 10:34:45 2022] GET /identity/v3/groups/56c1bbce28be4b57a92c54fae3188db0 => generated 226 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:34:45.258040 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9200925c-633c-40ec-b2a2-122d8aeffb29 None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:45.340657 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:45.375894 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:45.391853 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb4cdad-0dcb-450a-9f50-0a080b79f065 None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:45.427365 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9200925c-633c-40ec-b2a2-122d8aeffb29 None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:45.428840 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1312b71c-0349-4529-86e2-b286332b1109 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:45.430146 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1312b71c-0349-4529-86e2-b286332b1109 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:45.430552 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1312b71c-0349-4529-86e2-b286332b1109 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:45.431000 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1312b71c-0349-4529-86e2-b286332b1109 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:45.480712 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a203dc85-ccd1-48bd-bc6f-9304d4fc4f26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:45.481498 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a203dc85-ccd1-48bd-bc6f-9304d4fc4f26 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:45.481886 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a203dc85-ccd1-48bd-bc6f-9304d4fc4f26 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:45.482265 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a203dc85-ccd1-48bd-bc6f-9304d4fc4f26 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:45.483240 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a203dc85-ccd1-48bd-bc6f-9304d4fc4f26 None admin] RBAC: Adding query filter params (group.id=56c1bbce28be4b57a92c54fae3188db0, role.id=c3341f95596344309327efff99a20d5c, scope.project.id=33cf2502c8694374bd447d6304e658c2) {{(pid=86827) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:34:45.483655 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a203dc85-ccd1-48bd-bc6f-9304d4fc4f26 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:45.485352 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a203dc85-ccd1-48bd-bc6f-9304d4fc4f26 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:45.493297 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 226/435] 10.222.0.36 () {56 vars in 1376 bytes} [Wed Jul 27 10:34:45 2022] GET /identity/v3/role_assignments?group.id=56c1bbce28be4b57a92c54fae3188db0&scope.project.id=33cf2502c8694374bd447d6304e658c2&role.id=c3341f95596344309327efff99a20d5c => generated 611 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:34:45.517795 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1312b71c-0349-4529-86e2-b286332b1109 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:45.519593 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1312b71c-0349-4529-86e2-b286332b1109 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:45.614230 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:45.637077 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 210/436] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jul 27 10:34:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 27 10:34:45.642804 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a203dc85-ccd1-48bd-bc6f-9304d4fc4f26 None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:45.668135 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5111ac-fa06-443f-8ec8-786160cc70d8 None None] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:45.668428 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5111ac-fa06-443f-8ec8-786160cc70d8 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:45.668704 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5111ac-fa06-443f-8ec8-786160cc70d8 None None] PATH_INFO: `/` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:45.670054 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 211/437] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 10:34:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 10:34:45.675200 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0797b4-a67c-4086-ab01-af47a564f104 None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:45.675486 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0797b4-a67c-4086-ab01-af47a564f104 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:45.675760 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0797b4-a67c-4086-ab01-af47a564f104 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:45.700320 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-9f0797b4-a67c-4086-ab01-af47a564f104 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:34:45.701549 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-9f0797b4-a67c-4086-ab01-af47a564f104 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:34:45.785080 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a203dc85-ccd1-48bd-bc6f-9304d4fc4f26 None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:45.793926 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 212/438] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 10:34:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 27 10:34:45.817490 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0797b4-a67c-4086-ab01-af47a564f104 None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:45.817702 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:45.831274 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56366e2-08fa-409a-8710-9a270f3ae64c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:45.832286 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56366e2-08fa-409a-8710-9a270f3ae64c None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:45.832286 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56366e2-08fa-409a-8710-9a270f3ae64c None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:45.832530 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56366e2-08fa-409a-8710-9a270f3ae64c None admin] PATH_INFO: `/v3/projects/046cff0c48ee4df282871e4eda9cf939` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:45.833687 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56366e2-08fa-409a-8710-9a270f3ae64c None admin] RBAC: Authorizing `identity:get_project(project_id=046cff0c48ee4df282871e4eda9cf939)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:45.835264 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56366e2-08fa-409a-8710-9a270f3ae64c None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:45.837521 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 227/439] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Jul 27 10:34:45 2022] GET /identity/v3/projects/046cff0c48ee4df282871e4eda9cf939 => generated 302 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:34:45.843162 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56366e2-08fa-409a-8710-9a270f3ae64c None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:45.955654 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0797b4-a67c-4086-ab01-af47a564f104 None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:45.975429 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56366e2-08fa-409a-8710-9a270f3ae64c None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:45.977024 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:45.999275 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54a859e-c1a7-494a-b762-c94158df59f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:46.000145 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54a859e-c1a7-494a-b762-c94158df59f9 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:46.000685 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54a859e-c1a7-494a-b762-c94158df59f9 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:46.000960 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54a859e-c1a7-494a-b762-c94158df59f9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:46.012352 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54a859e-c1a7-494a-b762-c94158df59f9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 10:34:46.014581 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb0be52-ea18-4774-9445-48785a712c35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:46.015253 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb0be52-ea18-4774-9445-48785a712c35 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:46.015540 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb0be52-ea18-4774-9445-48785a712c35 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:46.015821 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb0be52-ea18-4774-9445-48785a712c35 None admin] PATH_INFO: `/v3/groups/56c1bbce28be4b57a92c54fae3188db0` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:46.016985 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb0be52-ea18-4774-9445-48785a712c35 None admin] RBAC: Authorizing `identity:get_group(group_id=56c1bbce28be4b57a92c54fae3188db0)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:46.017252 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application [None req-f54a859e-c1a7-494a-b762-c94158df59f9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 10:34:46.017252 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:34:46.017252 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:34:46.017252 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:34:46.017252 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:34:46.017252 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:34:46.017252 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:34:46.017252 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:34:46.017252 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:34:46.017252 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:34:46.017252 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:34:46.017252 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:34:46.017252 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 10:34:46.017252 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 10:34:46.017252 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 10:34:46.017252 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 10:34:46.018102 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 10:34:46.018102 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 10:34:46.018102 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 10:34:46.018102 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 10:34:46.018102 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:34:46.018102 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:34:46.018102 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:34:46.018102 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:34:46.018102 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:34:46.018102 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:34:46.018102 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:34:46.018102 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application with Lock( Jul 27 10:34:46.018102 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:34:46.018102 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:34:46.018102 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:34:46.018102 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:34:46.018961 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:34:46.018961 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:34:46.018961 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:34:46.018961 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:34:46.018961 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 10:34:46.018961 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 10:34:46.018961 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 10:34:46.018961 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 10:34:46.018961 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 10:34:46.018961 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 10:34:46.018961 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 10:34:46.018961 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Jul 27 10:34:46.020114 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb0be52-ea18-4774-9445-48785a712c35 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:46.023340 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 213/440] 10.222.0.36 () {56 vars in 1096 bytes} [Wed Jul 27 10:34:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 10:34:46.023973 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54a859e-c1a7-494a-b762-c94158df59f9 None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:46.024880 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 228/441] 10.222.0.36 () {56 vars in 1180 bytes} [Wed Jul 27 10:34:45 2022] GET /identity/v3/groups/56c1bbce28be4b57a92c54fae3188db0 => generated 226 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:34:46.025563 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb0be52-ea18-4774-9445-48785a712c35 None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:46.082545 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:46.129842 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb0be52-ea18-4774-9445-48785a712c35 None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:46.141333 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54a859e-c1a7-494a-b762-c94158df59f9 None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:46.164573 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa11ab07-14a8-44a8-8895-d3e3234c85ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:46.165192 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa11ab07-14a8-44a8-8895-d3e3234c85ed None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:46.165471 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa11ab07-14a8-44a8-8895-d3e3234c85ed None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:46.165742 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa11ab07-14a8-44a8-8895-d3e3234c85ed None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:46.166531 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa11ab07-14a8-44a8-8895-d3e3234c85ed None admin] RBAC: Adding query filter params (group.id=56c1bbce28be4b57a92c54fae3188db0, role.id=4c9fc0b3baed441da2126be41cc6b04a, scope.project.id=046cff0c48ee4df282871e4eda9cf939) {{(pid=86827) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:34:46.166872 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa11ab07-14a8-44a8-8895-d3e3234c85ed None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:46.168309 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa11ab07-14a8-44a8-8895-d3e3234c85ed None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:46.174268 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 229/442] 10.222.0.36 () {56 vars in 1376 bytes} [Wed Jul 27 10:34:46 2022] GET /identity/v3/role_assignments?group.id=56c1bbce28be4b57a92c54fae3188db0&scope.project.id=046cff0c48ee4df282871e4eda9cf939&role.id=4c9fc0b3baed441da2126be41cc6b04a => generated 611 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:34:46.187093 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1920535f-95f5-406e-bdcf-c2bf795d2e0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:46.187842 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1920535f-95f5-406e-bdcf-c2bf795d2e0a None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:46.188199 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1920535f-95f5-406e-bdcf-c2bf795d2e0a None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:46.188549 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1920535f-95f5-406e-bdcf-c2bf795d2e0a None admin] PATH_INFO: `/v3/roles` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:46.189376 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1920535f-95f5-406e-bdcf-c2bf795d2e0a None admin] RBAC: Adding query filter params (name=admin) {{(pid=86828) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:34:46.189755 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1920535f-95f5-406e-bdcf-c2bf795d2e0a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:46.191346 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1920535f-95f5-406e-bdcf-c2bf795d2e0a None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:46.200847 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65dc104-b11f-4db0-87a5-bbd254ad4aa3 None None] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:46.201239 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65dc104-b11f-4db0-87a5-bbd254ad4aa3 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:46.201410 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65dc104-b11f-4db0-87a5-bbd254ad4aa3 None None] PATH_INFO: `/` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:46.201738 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 214/443] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jul 27 10:34:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 10:34:46.204191 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 230/444] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 10:34:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 10:34:46.207119 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1920535f-95f5-406e-bdcf-c2bf795d2e0a None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:46.209746 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d51b65-c85d-4737-98ff-015d907d8f5f None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:46.210349 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d51b65-c85d-4737-98ff-015d907d8f5f None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:46.210658 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d51b65-c85d-4737-98ff-015d907d8f5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:46.232376 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-23d51b65-c85d-4737-98ff-015d907d8f5f None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:34:46.234131 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-23d51b65-c85d-4737-98ff-015d907d8f5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:34:46.250292 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:46.319003 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 231/445] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 10:34:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 27 10:34:46.321732 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1920535f-95f5-406e-bdcf-c2bf795d2e0a None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:46.339609 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d51b65-c85d-4737-98ff-015d907d8f5f None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:46.363785 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a9cfb2-4317-4487-9d01-4fcb90b547f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:46.364417 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a9cfb2-4317-4487-9d01-4fcb90b547f2 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:46.364701 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a9cfb2-4317-4487-9d01-4fcb90b547f2 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:46.364977 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a9cfb2-4317-4487-9d01-4fcb90b547f2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:46.399298 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:46.446096 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a9cfb2-4317-4487-9d01-4fcb90b547f2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:46.447750 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a9cfb2-4317-4487-9d01-4fcb90b547f2 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:46.470225 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d51b65-c85d-4737-98ff-015d907d8f5f None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:46.497740 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:46.510194 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b0e14b-bc34-4dbd-b96c-65b88ff7ed22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:46.510874 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b0e14b-bc34-4dbd-b96c-65b88ff7ed22 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:46.511152 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b0e14b-bc34-4dbd-b96c-65b88ff7ed22 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:46.511424 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b0e14b-bc34-4dbd-b96c-65b88ff7ed22 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:46.525394 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b0e14b-bc34-4dbd-b96c-65b88ff7ed22 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 10:34:46.530374 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application [None req-82b0e14b-bc34-4dbd-b96c-65b88ff7ed22 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 27 10:34:46.530374 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:34:46.530374 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:34:46.530374 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:34:46.530374 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:34:46.530374 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:34:46.530374 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:34:46.530374 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:34:46.530374 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:34:46.530374 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:34:46.530374 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:34:46.530374 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:34:46.530374 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 10:34:46.530374 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 10:34:46.530374 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 10:34:46.530374 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 10:34:46.531299 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 10:34:46.531299 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 10:34:46.531299 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 10:34:46.531299 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 10:34:46.531299 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:34:46.531299 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:34:46.531299 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:34:46.531299 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:34:46.531299 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:34:46.531299 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:34:46.531299 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:34:46.531299 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application with Lock( Jul 27 10:34:46.531299 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:34:46.531299 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:34:46.531299 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:34:46.531299 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:34:46.532083 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:34:46.532083 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:34:46.532083 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:34:46.532083 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:34:46.532083 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 10:34:46.532083 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 10:34:46.532083 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 10:34:46.532083 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 10:34:46.532083 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 10:34:46.532083 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 10:34:46.532083 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 27 10:34:46.532083 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Jul 27 10:34:46.532673 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 232/446] 10.222.0.36 () {56 vars in 1114 bytes} [Wed Jul 27 10:34:46 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 10:34:46.539179 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b0e14b-bc34-4dbd-b96c-65b88ff7ed22 None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:46.569250 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 215/447] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jul 27 10:34:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 10:34:46.574775 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a9cfb2-4317-4487-9d01-4fcb90b547f2 None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:46.639611 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b0e14b-bc34-4dbd-b96c-65b88ff7ed22 None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:46.650500 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:46.673206 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a9cfb2-4317-4487-9d01-4fcb90b547f2 None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:46.675554 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae3170a-1a44-40c3-b0d7-762fde99e283 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:46.676175 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae3170a-1a44-40c3-b0d7-762fde99e283 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:46.676457 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae3170a-1a44-40c3-b0d7-762fde99e283 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:46.676730 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae3170a-1a44-40c3-b0d7-762fde99e283 None admin] PATH_INFO: `/v3/roles` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:46.677443 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae3170a-1a44-40c3-b0d7-762fde99e283 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86827) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:34:46.677749 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae3170a-1a44-40c3-b0d7-762fde99e283 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:46.679164 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae3170a-1a44-40c3-b0d7-762fde99e283 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:46.686961 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 233/448] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Jul 27 10:34:46 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 10:34:46.692459 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae3170a-1a44-40c3-b0d7-762fde99e283 None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:46.706225 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c72f3d3-7695-4d81-bfa8-c9475dd94727 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:46.706898 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c72f3d3-7695-4d81-bfa8-c9475dd94727 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:46.707185 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c72f3d3-7695-4d81-bfa8-c9475dd94727 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:46.707467 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c72f3d3-7695-4d81-bfa8-c9475dd94727 None admin] PATH_INFO: `/v3/projects/eeee704de4024dd1b602d1a289cba407` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:46.708628 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c72f3d3-7695-4d81-bfa8-c9475dd94727 None admin] RBAC: Authorizing `identity:get_project(project_id=eeee704de4024dd1b602d1a289cba407)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:46.710065 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c72f3d3-7695-4d81-bfa8-c9475dd94727 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:46.712372 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 216/449] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Jul 27 10:34:46 2022] GET /identity/v3/projects/eeee704de4024dd1b602d1a289cba407 => generated 344 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 10:34:46.717775 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c72f3d3-7695-4d81-bfa8-c9475dd94727 None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:46.726076 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:46.793826 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae3170a-1a44-40c3-b0d7-762fde99e283 None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:46.816377 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c72f3d3-7695-4d81-bfa8-c9475dd94727 None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:46.826711 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd4bf3a-12e3-405a-b61c-e2e1e308d195 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:46.827340 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd4bf3a-12e3-405a-b61c-e2e1e308d195 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:46.827621 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd4bf3a-12e3-405a-b61c-e2e1e308d195 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:46.827900 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd4bf3a-12e3-405a-b61c-e2e1e308d195 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:46.850860 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c234732d-6ba9-4852-9553-c2ecc5d6ec24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:46.851504 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c234732d-6ba9-4852-9553-c2ecc5d6ec24 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:46.851793 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c234732d-6ba9-4852-9553-c2ecc5d6ec24 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:46.852076 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c234732d-6ba9-4852-9553-c2ecc5d6ec24 None admin] PATH_INFO: `/v3/groups/0926913586d841ea9dd4fdd2c32fcb7e` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:46.853243 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c234732d-6ba9-4852-9553-c2ecc5d6ec24 None admin] RBAC: Authorizing `identity:get_group(group_id=0926913586d841ea9dd4fdd2c32fcb7e)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:46.854711 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c234732d-6ba9-4852-9553-c2ecc5d6ec24 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:46.856429 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 217/450] 10.222.0.36 () {56 vars in 1180 bytes} [Wed Jul 27 10:34:46 2022] GET /identity/v3/groups/0926913586d841ea9dd4fdd2c32fcb7e => generated 229 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 10:34:46.862112 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c234732d-6ba9-4852-9553-c2ecc5d6ec24 None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:46.868999 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:46.896128 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd4bf3a-12e3-405a-b61c-e2e1e308d195 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:46.897596 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd4bf3a-12e3-405a-b61c-e2e1e308d195 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:46.935135 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:46.964584 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c234732d-6ba9-4852-9553-c2ecc5d6ec24 None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:46.996483 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 234/451] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jul 27 10:34:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 27 10:34:46.996816 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3737dc-c962-44a7-918a-c59941ffad34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:46.997454 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3737dc-c962-44a7-918a-c59941ffad34 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:46.997722 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3737dc-c962-44a7-918a-c59941ffad34 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:46.997998 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3737dc-c962-44a7-918a-c59941ffad34 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:46.998834 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3737dc-c962-44a7-918a-c59941ffad34 None admin] RBAC: Adding query filter params (group.id=0926913586d841ea9dd4fdd2c32fcb7e, role.id=21156560eb6b436bb567a40b4976f3f4, scope.project.id=eeee704de4024dd1b602d1a289cba407) {{(pid=86828) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:34:46.999144 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3737dc-c962-44a7-918a-c59941ffad34 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:47.000574 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3737dc-c962-44a7-918a-c59941ffad34 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:47.001928 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd4bf3a-12e3-405a-b61c-e2e1e308d195 None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:47.007816 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 218/452] 10.222.0.36 () {56 vars in 1376 bytes} [Wed Jul 27 10:34:46 2022] GET /identity/v3/role_assignments?group.id=0926913586d841ea9dd4fdd2c32fcb7e&scope.project.id=eeee704de4024dd1b602d1a289cba407&role.id=21156560eb6b436bb567a40b4976f3f4 => generated 611 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 10:34:47.073728 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:47.104626 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd4bf3a-12e3-405a-b61c-e2e1e308d195 None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:47.140034 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5191fb-6247-463d-a3a0-63669a683248 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:47.140695 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5191fb-6247-463d-a3a0-63669a683248 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:47.140976 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5191fb-6247-463d-a3a0-63669a683248 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:47.141256 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5191fb-6247-463d-a3a0-63669a683248 None admin] PATH_INFO: `/v3/projects/046cff0c48ee4df282871e4eda9cf939` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:47.142452 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5191fb-6247-463d-a3a0-63669a683248 None admin] RBAC: Authorizing `identity:get_project(project_id=046cff0c48ee4df282871e4eda9cf939)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:47.143933 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5191fb-6247-463d-a3a0-63669a683248 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:47.145824 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 235/453] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Jul 27 10:34:47 2022] GET /identity/v3/projects/046cff0c48ee4df282871e4eda9cf939 => generated 302 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 10:34:47.151197 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3737dc-c962-44a7-918a-c59941ffad34 None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:47.253780 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3737dc-c962-44a7-918a-c59941ffad34 None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:47.286347 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050fc2e4-54ff-4c25-848a-89b4cd2af422 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:47.287080 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050fc2e4-54ff-4c25-848a-89b4cd2af422 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:47.287365 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050fc2e4-54ff-4c25-848a-89b4cd2af422 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:47.287644 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050fc2e4-54ff-4c25-848a-89b4cd2af422 None admin] PATH_INFO: `/v3/groups/56c1bbce28be4b57a92c54fae3188db0` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:47.288804 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050fc2e4-54ff-4c25-848a-89b4cd2af422 None admin] RBAC: Authorizing `identity:get_group(group_id=56c1bbce28be4b57a92c54fae3188db0)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:47.290259 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050fc2e4-54ff-4c25-848a-89b4cd2af422 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:47.291880 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 219/454] 10.222.0.36 () {56 vars in 1180 bytes} [Wed Jul 27 10:34:47 2022] GET /identity/v3/groups/56c1bbce28be4b57a92c54fae3188db0 => generated 226 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:34:47.297790 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5191fb-6247-463d-a3a0-63669a683248 None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:47.395063 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5191fb-6247-463d-a3a0-63669a683248 None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:47.426786 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a8f97e-4589-491a-8876-3a0e976c8215 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:47.427369 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a8f97e-4589-491a-8876-3a0e976c8215 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:47.427654 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a8f97e-4589-491a-8876-3a0e976c8215 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:47.427928 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a8f97e-4589-491a-8876-3a0e976c8215 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:47.428706 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a8f97e-4589-491a-8876-3a0e976c8215 None admin] RBAC: Adding query filter params (group.id=56c1bbce28be4b57a92c54fae3188db0, role.id=c3341f95596344309327efff99a20d5c, scope.project.id=046cff0c48ee4df282871e4eda9cf939) {{(pid=86827) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:34:47.429505 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a8f97e-4589-491a-8876-3a0e976c8215 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:47.430937 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a8f97e-4589-491a-8876-3a0e976c8215 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:47.436482 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 236/455] 10.222.0.36 () {56 vars in 1376 bytes} [Wed Jul 27 10:34:47 2022] GET /identity/v3/role_assignments?group.id=56c1bbce28be4b57a92c54fae3188db0&scope.project.id=046cff0c48ee4df282871e4eda9cf939&role.id=c3341f95596344309327efff99a20d5c => generated 611 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:34:49.972847 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ab7a36-fc66-4ac3-a187-bae126308026 None None] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:49.973815 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ab7a36-fc66-4ac3-a187-bae126308026 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:49.974314 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ab7a36-fc66-4ac3-a187-bae126308026 None None] PATH_INFO: `/` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:49.976718 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 220/456] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 10:34:49 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 10:34:49.981882 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6e210e-e622-4c3a-9457-b22dd963ca72 None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:49.982752 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6e210e-e622-4c3a-9457-b22dd963ca72 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:49.983251 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6e210e-e622-4c3a-9457-b22dd963ca72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:49.996012 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:50.012078 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-2d6e210e-e622-4c3a-9457-b22dd963ca72 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:34:50.013276 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-2d6e210e-e622-4c3a-9457-b22dd963ca72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:34:50.097220 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 237/457] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 10:34:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 10:34:50.115601 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c663e3d8-310d-4277-8b1f-a0b10a6b5da6 None None] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:50.115891 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c663e3d8-310d-4277-8b1f-a0b10a6b5da6 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:50.116171 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c663e3d8-310d-4277-8b1f-a0b10a6b5da6 None None] PATH_INFO: `/` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:50.117560 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 221/458] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 10:34:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 10:34:50.123028 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf615e9a-c902-4669-98db-938d21580158 None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:50.123315 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf615e9a-c902-4669-98db-938d21580158 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:50.123595 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf615e9a-c902-4669-98db-938d21580158 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:50.130559 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c663e3d8-310d-4277-8b1f-a0b10a6b5da6 None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:50.147383 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:50.148890 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-cf615e9a-c902-4669-98db-938d21580158 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:34:50.151001 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-cf615e9a-c902-4669-98db-938d21580158 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:34:50.289409 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 238/459] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 10:34:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 27 10:34:50.312926 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf615e9a-c902-4669-98db-938d21580158 None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:50.335873 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c663e3d8-310d-4277-8b1f-a0b10a6b5da6 None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:50.399834 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db42887-1161-44ef-84ca-fda558340688 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:50.401198 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db42887-1161-44ef-84ca-fda558340688 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:50.401679 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db42887-1161-44ef-84ca-fda558340688 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:50.402160 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db42887-1161-44ef-84ca-fda558340688 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:50.418822 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:50.421641 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db42887-1161-44ef-84ca-fda558340688 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:50.423251 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db42887-1161-44ef-84ca-fda558340688 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:50.441704 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application [None req-2db42887-1161-44ef-84ca-fda558340688 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 10:34:50.441704 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:34:50.441704 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:34:50.441704 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:34:50.441704 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:34:50.441704 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:34:50.441704 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:34:50.441704 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:34:50.441704 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:34:50.441704 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:34:50.441704 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:34:50.441704 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:34:50.441704 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 10:34:50.441704 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 10:34:50.441704 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 10:34:50.442580 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 10:34:50.442580 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:34:50.442580 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:34:50.442580 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:34:50.442580 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:34:50.442580 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:34:50.442580 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:34:50.442580 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:34:50.442580 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application with Lock( Jul 27 10:34:50.442580 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:34:50.442580 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:34:50.442580 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:34:50.442580 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:34:50.442580 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:34:50.442580 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:34:50.442580 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:34:50.442580 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:34:50.443448 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 10:34:50.443448 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 10:34:50.443448 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 10:34:50.443448 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Jul 27 10:34:50.450660 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 222/460] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 10:34:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 316 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 10:34:50.452127 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f51124-a719-4acf-8c03-908d8e05ff70 None None] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:50.452633 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f51124-a719-4acf-8c03-908d8e05ff70 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:50.453157 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f51124-a719-4acf-8c03-908d8e05ff70 None None] PATH_INFO: `/` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:50.455150 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 223/461] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 10:34:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 10:34:50.456144 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f51124-a719-4acf-8c03-908d8e05ff70 None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:50.468085 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf615e9a-c902-4669-98db-938d21580158 None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:50.504343 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6d4b89-4976-4343-b4df-e6255c0aff33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:50.505496 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6d4b89-4976-4343-b4df-e6255c0aff33 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:50.506030 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6d4b89-4976-4343-b4df-e6255c0aff33 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:50.506500 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6d4b89-4976-4343-b4df-e6255c0aff33 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:50.522673 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6d4b89-4976-4343-b4df-e6255c0aff33 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:50.523666 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6d4b89-4976-4343-b4df-e6255c0aff33 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:50.543593 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application [None req-be6d4b89-4976-4343-b4df-e6255c0aff33 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 10:34:50.543593 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:34:50.543593 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:34:50.543593 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:34:50.543593 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:34:50.543593 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:34:50.543593 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:34:50.543593 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:34:50.543593 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:34:50.543593 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:34:50.543593 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:34:50.543593 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:34:50.543593 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 10:34:50.543593 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 10:34:50.543593 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 10:34:50.544649 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 10:34:50.544649 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:34:50.544649 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:34:50.544649 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:34:50.544649 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:34:50.544649 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:34:50.544649 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:34:50.544649 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:34:50.544649 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application with Lock( Jul 27 10:34:50.544649 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:34:50.544649 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:34:50.544649 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:34:50.544649 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:34:50.544649 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:34:50.544649 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:34:50.544649 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:34:50.544649 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:34:50.546432 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 10:34:50.546432 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 10:34:50.546432 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 10:34:50.546432 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Jul 27 10:34:50.546732 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:50.549355 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 239/462] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 10:34:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 10:34:50.550969 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35379899-1a8d-4c4d-b778-41f85f837ce8 None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:50.551392 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35379899-1a8d-4c4d-b778-41f85f837ce8 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:50.551819 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35379899-1a8d-4c4d-b778-41f85f837ce8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:50.576359 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-35379899-1a8d-4c4d-b778-41f85f837ce8 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:34:50.578355 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-35379899-1a8d-4c4d-b778-41f85f837ce8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:34:50.599203 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f51124-a719-4acf-8c03-908d8e05ff70 None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:50.644741 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375b336d-ee41-4981-9005-cf78b386e210 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:50.645405 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375b336d-ee41-4981-9005-cf78b386e210 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:50.645688 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375b336d-ee41-4981-9005-cf78b386e210 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:50.645992 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375b336d-ee41-4981-9005-cf78b386e210 None admin] PATH_INFO: `/v3/domains` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:50.646885 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375b336d-ee41-4981-9005-cf78b386e210 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86828) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:34:50.647193 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375b336d-ee41-4981-9005-cf78b386e210 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:50.648734 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375b336d-ee41-4981-9005-cf78b386e210 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:50.667707 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 224/463] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 10:34:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:34:50.669058 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375b336d-ee41-4981-9005-cf78b386e210 None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:50.670626 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 240/464] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 10:34:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 10:34:50.675501 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35379899-1a8d-4c4d-b778-41f85f837ce8 None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:50.742131 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:50.796593 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375b336d-ee41-4981-9005-cf78b386e210 None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:50.812491 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35379899-1a8d-4c4d-b778-41f85f837ce8 None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:50.817145 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:50.840884 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80f25fc-2270-4316-9923-adcdb73f3f8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:50.841513 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80f25fc-2270-4316-9923-adcdb73f3f8c None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:50.841801 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80f25fc-2270-4316-9923-adcdb73f3f8c None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:50.842083 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80f25fc-2270-4316-9923-adcdb73f3f8c None admin] PATH_INFO: `/v3/domains` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:50.842815 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80f25fc-2270-4316-9923-adcdb73f3f8c None admin] RBAC: Adding query filter params (name=Default) {{(pid=86828) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:34:50.843126 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80f25fc-2270-4316-9923-adcdb73f3f8c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:50.844509 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80f25fc-2270-4316-9923-adcdb73f3f8c None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:50.854062 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103a2adf-aa85-4746-be77-a1dbd9c76c58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:50.854811 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103a2adf-aa85-4746-be77-a1dbd9c76c58 None admin] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:50.855101 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103a2adf-aa85-4746-be77-a1dbd9c76c58 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:50.855378 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103a2adf-aa85-4746-be77-a1dbd9c76c58 None admin] PATH_INFO: `/v3/users` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:50.856062 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103a2adf-aa85-4746-be77-a1dbd9c76c58 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:50.856513 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 225/465] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 10:34:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:34:50.857208 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80f25fc-2270-4316-9923-adcdb73f3f8c None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:50.857588 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103a2adf-aa85-4746-be77-a1dbd9c76c58 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:50.883717 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 241/466] 10.222.0.36 () {60 vars in 1132 bytes} [Wed Jul 27 10:34:50 2022] POST /identity/v3/users => generated 246 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 10:34:50.885716 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fde527-c42c-4757-a4c5-eb75fb60e631 None None] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:50.886157 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fde527-c42c-4757-a4c5-eb75fb60e631 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:50.886583 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fde527-c42c-4757-a4c5-eb75fb60e631 None None] PATH_INFO: `/` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:50.888314 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 242/467] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 10:34:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 10:34:50.891590 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fde527-c42c-4757-a4c5-eb75fb60e631 None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:50.987045 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80f25fc-2270-4316-9923-adcdb73f3f8c None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:51.029467 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fde527-c42c-4757-a4c5-eb75fb60e631 None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:51.030009 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144197e7-bd12-49f3-86f0-53d307051d6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:51.030141 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144197e7-bd12-49f3-86f0-53d307051d6f None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:51.030554 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144197e7-bd12-49f3-86f0-53d307051d6f None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:51.030949 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144197e7-bd12-49f3-86f0-53d307051d6f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:51.032654 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:51.047536 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144197e7-bd12-49f3-86f0-53d307051d6f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:51.050565 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144197e7-bd12-49f3-86f0-53d307051d6f None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:51.072094 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application [None req-144197e7-bd12-49f3-86f0-53d307051d6f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 10:34:51.072094 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:34:51.072094 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:34:51.072094 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:34:51.072094 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:34:51.072094 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:34:51.072094 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:34:51.072094 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:34:51.072094 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:34:51.072094 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:34:51.072094 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:34:51.072094 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:34:51.072094 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 10:34:51.072094 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 10:34:51.072094 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 10:34:51.073183 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 10:34:51.073183 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:34:51.073183 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:34:51.073183 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:34:51.073183 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:34:51.073183 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:34:51.073183 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:34:51.073183 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:34:51.073183 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application with Lock( Jul 27 10:34:51.073183 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:34:51.073183 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:34:51.073183 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:34:51.073183 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:34:51.073183 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:34:51.073183 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:34:51.073183 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:34:51.073183 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:34:51.074185 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 10:34:51.074185 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 10:34:51.074185 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 10:34:51.074185 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Jul 27 10:34:51.075229 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 226/468] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 10:34:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 10:34:51.076547 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c8af4e-19d2-4079-941c-fb113ef9de2c None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:51.076843 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c8af4e-19d2-4079-941c-fb113ef9de2c None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:51.077120 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c8af4e-19d2-4079-941c-fb113ef9de2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:51.084770 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f6e9a6-72c5-4504-8185-f89d93f865cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:51.085568 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f6e9a6-72c5-4504-8185-f89d93f865cf None admin] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:51.085924 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f6e9a6-72c5-4504-8185-f89d93f865cf None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:51.086295 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f6e9a6-72c5-4504-8185-f89d93f865cf None admin] PATH_INFO: `/v3/users` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:51.087108 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f6e9a6-72c5-4504-8185-f89d93f865cf None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:51.088546 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f6e9a6-72c5-4504-8185-f89d93f865cf None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:51.096141 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-d3c8af4e-19d2-4079-941c-fb113ef9de2c None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:34:51.097614 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-d3c8af4e-19d2-4079-941c-fb113ef9de2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:34:51.115464 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 243/469] 10.222.0.36 () {60 vars in 1132 bytes} [Wed Jul 27 10:34:50 2022] POST /identity/v3/users => generated 248 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 10:34:51.116306 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f6e9a6-72c5-4504-8185-f89d93f865cf None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:51.133677 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:51.197974 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 227/470] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 10:34:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 10:34:51.222692 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c8af4e-19d2-4079-941c-fb113ef9de2c None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:51.245287 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f6e9a6-72c5-4504-8185-f89d93f865cf None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:51.284707 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18dd816f-d9ff-49b5-848d-c36cc1ec2f23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:51.285440 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dd816f-d9ff-49b5-848d-c36cc1ec2f23 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:51.285729 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dd816f-d9ff-49b5-848d-c36cc1ec2f23 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:51.286007 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dd816f-d9ff-49b5-848d-c36cc1ec2f23 None admin] PATH_INFO: `/v3/domains` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:51.286800 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18dd816f-d9ff-49b5-848d-c36cc1ec2f23 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86827) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:34:51.287117 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18dd816f-d9ff-49b5-848d-c36cc1ec2f23 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:51.289301 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18dd816f-d9ff-49b5-848d-c36cc1ec2f23 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:51.312654 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 244/471] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 10:34:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:34:51.320077 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18dd816f-d9ff-49b5-848d-c36cc1ec2f23 None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:51.343608 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c8af4e-19d2-4079-941c-fb113ef9de2c None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:51.356356 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:51.389178 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c111de-5ec0-4f8d-8246-696f50d41acb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:51.389928 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c111de-5ec0-4f8d-8246-696f50d41acb None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:51.390209 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c111de-5ec0-4f8d-8246-696f50d41acb None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:51.390501 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c111de-5ec0-4f8d-8246-696f50d41acb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:51.402153 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c111de-5ec0-4f8d-8246-696f50d41acb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:51.403718 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c111de-5ec0-4f8d-8246-696f50d41acb None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:51.420832 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application [None req-10c111de-5ec0-4f8d-8246-696f50d41acb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 10:34:51.420832 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:34:51.420832 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:34:51.420832 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:34:51.420832 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:34:51.420832 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:34:51.420832 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:34:51.420832 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:34:51.420832 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:34:51.420832 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:34:51.420832 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:34:51.420832 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:34:51.420832 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 10:34:51.420832 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 10:34:51.420832 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 10:34:51.421703 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 10:34:51.421703 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:34:51.421703 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:34:51.421703 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:34:51.421703 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:34:51.421703 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:34:51.421703 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:34:51.421703 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:34:51.421703 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application with Lock( Jul 27 10:34:51.421703 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:34:51.421703 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:34:51.421703 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:34:51.421703 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:34:51.421703 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:34:51.421703 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:34:51.421703 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:34:51.421703 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:34:51.422999 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 10:34:51.422999 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 10:34:51.422999 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 10:34:51.422999 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Jul 27 10:34:51.422999 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 228/472] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 10:34:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 10:34:51.428009 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c111de-5ec0-4f8d-8246-696f50d41acb None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:51.445624 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:51.462903 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18dd816f-d9ff-49b5-848d-c36cc1ec2f23 None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:51.528548 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8423868e-09d6-4d1b-a696-18e4bc2ba567 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:51.529191 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8423868e-09d6-4d1b-a696-18e4bc2ba567 None admin] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:51.529476 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8423868e-09d6-4d1b-a696-18e4bc2ba567 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:51.529753 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8423868e-09d6-4d1b-a696-18e4bc2ba567 None admin] PATH_INFO: `/v3/users` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:51.530445 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8423868e-09d6-4d1b-a696-18e4bc2ba567 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:51.531933 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8423868e-09d6-4d1b-a696-18e4bc2ba567 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:51.559027 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 245/473] 10.222.0.36 () {60 vars in 1132 bytes} [Wed Jul 27 10:34:51 2022] POST /identity/v3/users => generated 248 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 10:34:51.561494 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f4d0cb-4ac2-44c6-95c8-4ff19b10b9e6 None None] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:51.563584 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f4d0cb-4ac2-44c6-95c8-4ff19b10b9e6 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:51.563584 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f4d0cb-4ac2-44c6-95c8-4ff19b10b9e6 None None] PATH_INFO: `/` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:51.566648 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 246/474] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 10:34:51 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 10:34:51.572443 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c111de-5ec0-4f8d-8246-696f50d41acb None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:51.575590 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da748ae5-6b1e-4478-92fa-102eadabe395 None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:51.575892 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da748ae5-6b1e-4478-92fa-102eadabe395 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:51.576155 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da748ae5-6b1e-4478-92fa-102eadabe395 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:51.605698 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-da748ae5-6b1e-4478-92fa-102eadabe395 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:34:51.606954 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-da748ae5-6b1e-4478-92fa-102eadabe395 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:34:51.662045 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b47330-f645-4824-99d0-a9daf925d1d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:51.662717 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b47330-f645-4824-99d0-a9daf925d1d6 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:51.663007 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b47330-f645-4824-99d0-a9daf925d1d6 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:51.663357 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b47330-f645-4824-99d0-a9daf925d1d6 None admin] PATH_INFO: `/v3/domains` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:51.664598 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b47330-f645-4824-99d0-a9daf925d1d6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86828) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:34:51.664911 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b47330-f645-4824-99d0-a9daf925d1d6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:51.666353 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b47330-f645-4824-99d0-a9daf925d1d6 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:51.693798 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 229/475] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 10:34:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 10:34:51.697937 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b47330-f645-4824-99d0-a9daf925d1d6 None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:51.732110 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 247/476] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 10:34:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 27 10:34:51.763048 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da748ae5-6b1e-4478-92fa-102eadabe395 None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:51.766210 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:51.811954 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:51.844589 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b47330-f645-4824-99d0-a9daf925d1d6 None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:51.892029 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da748ae5-6b1e-4478-92fa-102eadabe395 None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:51.893931 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0507db-4169-4210-b263-555b4c4e01c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:51.894683 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0507db-4169-4210-b263-555b4c4e01c8 None admin] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:51.895875 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0507db-4169-4210-b263-555b4c4e01c8 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:51.896249 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0507db-4169-4210-b263-555b4c4e01c8 None admin] PATH_INFO: `/v3/users` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:51.897044 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0507db-4169-4210-b263-555b4c4e01c8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:51.899141 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0507db-4169-4210-b263-555b4c4e01c8 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:51.928683 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 230/477] 10.222.0.36 () {60 vars in 1132 bytes} [Wed Jul 27 10:34:51 2022] POST /identity/v3/users => generated 249 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 10:34:51.933796 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d8ab89-e1bd-4531-83d0-41205a303f81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:51.934546 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d8ab89-e1bd-4531-83d0-41205a303f81 None admin] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:51.934870 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d8ab89-e1bd-4531-83d0-41205a303f81 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:51.935138 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d8ab89-e1bd-4531-83d0-41205a303f81 None admin] PATH_INFO: `/v3/roles` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:51.935983 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d8ab89-e1bd-4531-83d0-41205a303f81 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:51.937424 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d8ab89-e1bd-4531-83d0-41205a303f81 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:51.949348 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.sql.core [None req-79d8ab89-e1bd-4531-83d0-41205a303f81 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 27 10:34:51.949348 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 27 10:34:51.949348 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [parameters: {'id': 'cb6ac34612874c5b8ff2dd8e0da9ebba', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 27 10:34:51.949348 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=86827) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 27 10:34:51.968103 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application [None req-79d8ab89-e1bd-4531-83d0-41205a303f81 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 27 10:34:51.968103 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:34:51.968103 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 27 10:34:51.968103 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 27 10:34:51.968103 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 27 10:34:51.968103 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 27 10:34:51.968103 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 27 10:34:51.968103 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application result = self._query(query) Jul 27 10:34:51.968103 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 27 10:34:51.968103 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application conn.query(q) Jul 27 10:34:51.968103 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 27 10:34:51.968103 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 27 10:34:51.968103 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 27 10:34:51.968103 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application result.read() Jul 27 10:34:51.968103 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 27 10:34:51.969423 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 27 10:34:51.969423 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 27 10:34:51.969423 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 27 10:34:51.969423 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 27 10:34:51.969423 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 27 10:34:51.969423 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 27 10:34:51.969423 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 27 10:34:51.969423 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 27 10:34:51.969423 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Jul 27 10:34:51.969423 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 27 10:34:51.969423 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Jul 27 10:34:51.969423 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:34:51.969423 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 27 10:34:51.969423 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 27 10:34:51.969423 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jul 27 10:34:51.969423 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return ref.to_dict() Jul 27 10:34:51.969423 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 27 10:34:51.969423 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application next(self.gen) Jul 27 10:34:51.970290 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 27 10:34:51.970290 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application yield resource Jul 27 10:34:51.970290 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 27 10:34:51.970290 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application next(self.gen) Jul 27 10:34:51.970290 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 27 10:34:51.970290 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application self.session.rollback() Jul 27 10:34:51.970290 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 27 10:34:51.970290 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application self.force_reraise() Jul 27 10:34:51.970290 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 27 10:34:51.970290 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application raise self.value Jul 27 10:34:51.970290 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 27 10:34:51.970290 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 27 10:34:51.970290 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 27 10:34:51.970290 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application session.commit() Jul 27 10:34:51.970290 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 27 10:34:51.970290 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 27 10:34:51.970290 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 27 10:34:51.971320 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application self._prepare_impl() Jul 27 10:34:51.971320 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 27 10:34:51.971320 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application self.session.flush() Jul 27 10:34:51.971320 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 27 10:34:51.971320 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application self._flush(objects) Jul 27 10:34:51.971320 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 27 10:34:51.971320 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 27 10:34:51.971320 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 27 10:34:51.971320 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application compat.raise_( Jul 27 10:34:51.971320 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 27 10:34:51.971320 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application raise exception Jul 27 10:34:51.971320 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 27 10:34:51.971320 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application flush_context.execute() Jul 27 10:34:51.971320 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 27 10:34:51.971320 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application rec.execute(self) Jul 27 10:34:51.971320 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 27 10:34:51.971320 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 27 10:34:51.972383 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 27 10:34:51.972383 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 27 10:34:51.972383 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 27 10:34:51.972383 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 27 10:34:51.972383 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 27 10:34:51.972383 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 27 10:34:51.972383 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 27 10:34:51.972383 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 27 10:34:51.972383 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 27 10:34:51.972383 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 27 10:34:51.972383 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 27 10:34:51.972383 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 27 10:34:51.972383 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 27 10:34:51.972383 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 27 10:34:51.972383 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 27 10:34:51.972383 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application raise exception Jul 27 10:34:51.973439 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 27 10:34:51.973439 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 27 10:34:51.973439 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 27 10:34:51.973439 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 27 10:34:51.973439 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 27 10:34:51.973439 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application result = self._query(query) Jul 27 10:34:51.973439 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 27 10:34:51.973439 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application conn.query(q) Jul 27 10:34:51.973439 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 27 10:34:51.973439 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 27 10:34:51.973439 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 27 10:34:51.973439 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application result.read() Jul 27 10:34:51.973439 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 27 10:34:51.973439 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 27 10:34:51.973439 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 27 10:34:51.973439 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 27 10:34:51.974409 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 27 10:34:51.974409 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 27 10:34:51.974409 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 27 10:34:51.974409 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 27 10:34:51.974409 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 27 10:34:51.974409 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 27 10:34:51.974409 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application [parameters: {'id': 'cb6ac34612874c5b8ff2dd8e0da9ebba', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 27 10:34:51.974409 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 27 10:34:51.974409 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Jul 27 10:34:51.974409 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 10:34:51.974409 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Jul 27 10:34:51.974409 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:34:51.974409 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:34:51.974409 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:34:51.974409 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:34:51.974409 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:34:51.975435 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:34:51.975435 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:34:51.975435 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:34:51.975435 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:34:51.975435 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:34:51.975435 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:34:51.975435 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jul 27 10:34:51.975435 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jul 27 10:34:51.975435 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:34:51.975435 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:34:51.975435 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jul 27 10:34:51.975435 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jul 27 10:34:51.975435 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 27 10:34:51.975435 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 27 10:34:51.975435 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 27 10:34:51.975435 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Jul 27 10:34:51.977313 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 248/478] 10.222.0.36 () {60 vars in 1131 bytes} [Wed Jul 27 10:34:51 2022] POST /identity/v3/roles => generated 144 bytes in 207 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jul 27 10:34:51.986251 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0507db-4169-4210-b263-555b4c4e01c8 None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:52.143242 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0507db-4169-4210-b263-555b4c4e01c8 None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:52.173395 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:52.183807 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57f6652-764c-4ba1-8dfe-29b1d6299fcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:52.184455 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57f6652-764c-4ba1-8dfe-29b1d6299fcf None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:52.184742 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57f6652-764c-4ba1-8dfe-29b1d6299fcf None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:52.185019 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57f6652-764c-4ba1-8dfe-29b1d6299fcf None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:52.196575 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57f6652-764c-4ba1-8dfe-29b1d6299fcf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 10:34:52.202068 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application [None req-e57f6652-764c-4ba1-8dfe-29b1d6299fcf None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 27 10:34:52.202068 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:34:52.202068 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:34:52.202068 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:34:52.202068 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:34:52.202068 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:34:52.202068 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:34:52.202068 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:34:52.202068 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:34:52.202068 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:34:52.202068 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:34:52.202068 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:34:52.202068 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 10:34:52.202068 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 10:34:52.202068 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 10:34:52.202068 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 10:34:52.202971 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 10:34:52.202971 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 10:34:52.202971 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 10:34:52.202971 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 10:34:52.202971 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:34:52.202971 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:34:52.202971 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:34:52.202971 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:34:52.202971 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:34:52.202971 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:34:52.202971 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:34:52.202971 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application with Lock( Jul 27 10:34:52.202971 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:34:52.202971 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:34:52.202971 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:34:52.202971 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:34:52.203778 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:34:52.203778 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:34:52.203778 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:34:52.203778 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:34:52.203778 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 10:34:52.203778 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 10:34:52.203778 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 10:34:52.203778 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 10:34:52.203778 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 10:34:52.203778 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 10:34:52.203778 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 27 10:34:52.203778 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Jul 27 10:34:52.203778 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 231/479] 10.222.0.36 () {56 vars in 1114 bytes} [Wed Jul 27 10:34:51 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 10:34:52.211138 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d8ab89-e1bd-4531-83d0-41205a303f81 None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:52.282670 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:52.317691 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d8ab89-e1bd-4531-83d0-41205a303f81 None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:52.353684 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df3cf11-13bc-420f-b6e1-0fbad699c800 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:52.354312 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df3cf11-13bc-420f-b6e1-0fbad699c800 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:52.354618 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df3cf11-13bc-420f-b6e1-0fbad699c800 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:52.354900 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df3cf11-13bc-420f-b6e1-0fbad699c800 None admin] PATH_INFO: `/v3/roles` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:52.355630 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df3cf11-13bc-420f-b6e1-0fbad699c800 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86827) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:34:52.355939 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df3cf11-13bc-420f-b6e1-0fbad699c800 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:52.357331 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df3cf11-13bc-420f-b6e1-0fbad699c800 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:52.366173 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 249/480] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Jul 27 10:34:52 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 10:34:53.643619 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef2c4ac-ec00-4685-a141-92f808508e7f None None] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:53.644158 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef2c4ac-ec00-4685-a141-92f808508e7f None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:53.644158 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef2c4ac-ec00-4685-a141-92f808508e7f None None] PATH_INFO: `/` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:53.645522 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 232/481] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 10:34:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 10:34:53.660428 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f59bed5-ba6e-44dd-a6ba-8a9ddcc6eed7 None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:53.660964 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f59bed5-ba6e-44dd-a6ba-8a9ddcc6eed7 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:53.660964 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f59bed5-ba6e-44dd-a6ba-8a9ddcc6eed7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:53.670283 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d3ba7f-0e26-4b20-a5c4-d25a9eb66c91 None None] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:53.670937 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d3ba7f-0e26-4b20-a5c4-d25a9eb66c91 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:53.671406 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d3ba7f-0e26-4b20-a5c4-d25a9eb66c91 None None] PATH_INFO: `/` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:53.673322 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 233/482] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 10:34:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 10:34:53.678774 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197727aa-b1c2-4cc1-9828-423f808da148 None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:53.679352 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197727aa-b1c2-4cc1-9828-423f808da148 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:53.679818 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197727aa-b1c2-4cc1-9828-423f808da148 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:53.688138 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-7f59bed5-ba6e-44dd-a6ba-8a9ddcc6eed7 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:34:53.689344 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-7f59bed5-ba6e-44dd-a6ba-8a9ddcc6eed7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:34:53.715941 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-197727aa-b1c2-4cc1-9828-423f808da148 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:34:53.717673 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-197727aa-b1c2-4cc1-9828-423f808da148 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:34:53.804891 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 250/483] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 10:34:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 10:34:53.834940 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f59bed5-ba6e-44dd-a6ba-8a9ddcc6eed7 None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:53.853650 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 234/484] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 10:34:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 27 10:34:53.881526 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197727aa-b1c2-4cc1-9828-423f808da148 None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:53.908594 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:53.958622 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f59bed5-ba6e-44dd-a6ba-8a9ddcc6eed7 None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:53.998274 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141c6fb9-dfa6-464a-ad28-dbda74bcb110 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:53.998975 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141c6fb9-dfa6-464a-ad28-dbda74bcb110 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:53.999260 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141c6fb9-dfa6-464a-ad28-dbda74bcb110 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:53.999534 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141c6fb9-dfa6-464a-ad28-dbda74bcb110 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:54.025260 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141c6fb9-dfa6-464a-ad28-dbda74bcb110 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 10:34:54.035827 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application [None req-141c6fb9-dfa6-464a-ad28-dbda74bcb110 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 27 10:34:54.035827 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:34:54.035827 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:34:54.035827 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:34:54.035827 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:34:54.035827 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:34:54.035827 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:34:54.035827 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:34:54.035827 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:34:54.035827 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:34:54.035827 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:34:54.035827 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:34:54.035827 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 10:34:54.035827 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 10:34:54.035827 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 10:34:54.035827 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 10:34:54.036728 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 10:34:54.036728 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 10:34:54.036728 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 10:34:54.036728 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 10:34:54.036728 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:34:54.036728 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:34:54.036728 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:34:54.036728 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:34:54.036728 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:34:54.036728 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:34:54.036728 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:34:54.036728 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application with Lock( Jul 27 10:34:54.036728 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:34:54.036728 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:34:54.036728 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:34:54.036728 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:34:54.037505 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:34:54.037505 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:34:54.037505 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:34:54.037505 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:34:54.037505 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 10:34:54.037505 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 10:34:54.037505 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 10:34:54.037505 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 10:34:54.037505 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 10:34:54.037505 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 10:34:54.037505 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 27 10:34:54.037505 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Jul 27 10:34:54.037505 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 251/485] 10.222.0.36 () {56 vars in 1102 bytes} [Wed Jul 27 10:34:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 10:34:54.042984 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141c6fb9-dfa6-464a-ad28-dbda74bcb110 None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:54.093075 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197727aa-b1c2-4cc1-9828-423f808da148 None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:54.111410 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:54.140541 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00110820-d2c5-47ac-bcb9-affd92dbb926 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:54.141186 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00110820-d2c5-47ac-bcb9-affd92dbb926 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:54.141466 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00110820-d2c5-47ac-bcb9-affd92dbb926 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:54.141736 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00110820-d2c5-47ac-bcb9-affd92dbb926 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:54.162461 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00110820-d2c5-47ac-bcb9-affd92dbb926 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 10:34:54.167428 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application [None req-00110820-d2c5-47ac-bcb9-affd92dbb926 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 27 10:34:54.167428 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:34:54.167428 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:34:54.167428 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:34:54.167428 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:34:54.167428 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:34:54.167428 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:34:54.167428 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:34:54.167428 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:34:54.167428 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:34:54.167428 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:34:54.167428 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:34:54.167428 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 10:34:54.167428 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 10:34:54.167428 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 10:34:54.167428 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 10:34:54.168287 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 10:34:54.168287 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 10:34:54.168287 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 10:34:54.168287 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 10:34:54.168287 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:34:54.168287 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:34:54.168287 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:34:54.168287 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:34:54.168287 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:34:54.168287 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:34:54.168287 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:34:54.168287 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application with Lock( Jul 27 10:34:54.168287 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:34:54.168287 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:34:54.168287 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:34:54.168287 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:34:54.169088 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:34:54.169088 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:34:54.169088 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:34:54.169088 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:34:54.169088 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 10:34:54.169088 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 10:34:54.169088 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 10:34:54.169088 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 10:34:54.169088 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 10:34:54.169088 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 10:34:54.169088 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 27 10:34:54.169088 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Jul 27 10:34:54.169088 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 235/486] 10.222.0.36 () {56 vars in 1102 bytes} [Wed Jul 27 10:34:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 10:34:54.174801 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b018a0ff-9263-46cc-bab3-7a8ac9b8e2f3 None None] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:54.174801 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b018a0ff-9263-46cc-bab3-7a8ac9b8e2f3 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:54.174801 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b018a0ff-9263-46cc-bab3-7a8ac9b8e2f3 None None] PATH_INFO: `/` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:54.174801 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 236/487] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 10:34:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 10:34:54.176773 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b018a0ff-9263-46cc-bab3-7a8ac9b8e2f3 None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:54.219946 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141c6fb9-dfa6-464a-ad28-dbda74bcb110 None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:54.248152 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:54.268392 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc744d5-5f0f-4f4d-837b-f3c45db97ac0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:54.269810 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc744d5-5f0f-4f4d-837b-f3c45db97ac0 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:54.270290 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc744d5-5f0f-4f4d-837b-f3c45db97ac0 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:54.271173 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc744d5-5f0f-4f4d-837b-f3c45db97ac0 None admin] PATH_INFO: `/v3/roles` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:54.272394 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc744d5-5f0f-4f4d-837b-f3c45db97ac0 None admin] RBAC: Adding query filter params (name=service) {{(pid=86827) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:34:54.272894 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc744d5-5f0f-4f4d-837b-f3c45db97ac0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:54.275971 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc744d5-5f0f-4f4d-837b-f3c45db97ac0 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:54.287508 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 252/488] 10.222.0.36 () {56 vars in 1103 bytes} [Wed Jul 27 10:34:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:34:54.289747 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1320ce7f-1a93-4f92-ac88-6d4b4472e39f None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:54.290498 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1320ce7f-1a93-4f92-ac88-6d4b4472e39f None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:54.291041 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1320ce7f-1a93-4f92-ac88-6d4b4472e39f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:54.296344 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b018a0ff-9263-46cc-bab3-7a8ac9b8e2f3 None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:54.314221 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-1320ce7f-1a93-4f92-ac88-6d4b4472e39f None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:34:54.316362 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-1320ce7f-1a93-4f92-ac88-6d4b4472e39f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:34:54.339736 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa262cf-47ac-43f8-8fe7-33ee200727e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:54.341098 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa262cf-47ac-43f8-8fe7-33ee200727e1 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:54.341622 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa262cf-47ac-43f8-8fe7-33ee200727e1 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:54.342120 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa262cf-47ac-43f8-8fe7-33ee200727e1 None admin] PATH_INFO: `/v3/roles` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:54.343651 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa262cf-47ac-43f8-8fe7-33ee200727e1 None admin] RBAC: Adding query filter params (name=service) {{(pid=86828) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:34:54.344171 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa262cf-47ac-43f8-8fe7-33ee200727e1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:54.346290 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa262cf-47ac-43f8-8fe7-33ee200727e1 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:54.362481 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 237/489] 10.222.0.36 () {56 vars in 1103 bytes} [Wed Jul 27 10:34:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:34:54.364468 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa262cf-47ac-43f8-8fe7-33ee200727e1 None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:54.403020 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 253/490] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 10:34:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 10:34:54.403922 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1320ce7f-1a93-4f92-ac88-6d4b4472e39f None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:54.416846 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:54.483778 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa262cf-47ac-43f8-8fe7-33ee200727e1 None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:54.519250 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf87712-344b-4a28-9fe5-a8cd7f35b436 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:54.519641 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf87712-344b-4a28-9fe5-a8cd7f35b436 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:54.519925 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf87712-344b-4a28-9fe5-a8cd7f35b436 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:54.520194 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf87712-344b-4a28-9fe5-a8cd7f35b436 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:54.531209 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf87712-344b-4a28-9fe5-a8cd7f35b436 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:54.532659 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf87712-344b-4a28-9fe5-a8cd7f35b436 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:54.540563 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1320ce7f-1a93-4f92-ac88-6d4b4472e39f None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:54.548111 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application [None req-baf87712-344b-4a28-9fe5-a8cd7f35b436 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 10:34:54.548111 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:34:54.548111 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:34:54.548111 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:34:54.548111 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:34:54.548111 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:34:54.548111 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:34:54.548111 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:34:54.548111 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:34:54.548111 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:34:54.548111 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:34:54.548111 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:34:54.548111 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 10:34:54.548111 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 10:34:54.548111 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 10:34:54.548935 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 10:34:54.548935 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:34:54.548935 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:34:54.548935 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:34:54.548935 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:34:54.548935 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:34:54.548935 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:34:54.548935 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:34:54.548935 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application with Lock( Jul 27 10:34:54.548935 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:34:54.548935 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:34:54.548935 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:34:54.548935 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:34:54.548935 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:34:54.548935 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:34:54.548935 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:34:54.548935 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:34:54.549784 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 10:34:54.549784 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 10:34:54.549784 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 10:34:54.549784 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Jul 27 10:34:54.549784 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 238/491] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 10:34:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 10:34:54.550566 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf87712-344b-4a28-9fe5-a8cd7f35b436 None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:54.558533 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:54.582903 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edde9252-38fd-465a-a669-07e62fdbd83e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:54.583913 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edde9252-38fd-465a-a669-07e62fdbd83e None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:54.584500 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edde9252-38fd-465a-a669-07e62fdbd83e None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:54.585285 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edde9252-38fd-465a-a669-07e62fdbd83e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:54.597142 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edde9252-38fd-465a-a669-07e62fdbd83e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:54.599004 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edde9252-38fd-465a-a669-07e62fdbd83e None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:54.616867 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application [None req-edde9252-38fd-465a-a669-07e62fdbd83e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 10:34:54.616867 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:34:54.616867 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:34:54.616867 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:34:54.616867 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:34:54.616867 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:34:54.616867 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:34:54.616867 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:34:54.616867 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:34:54.616867 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:34:54.616867 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:34:54.616867 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:34:54.616867 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 10:34:54.616867 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 10:34:54.616867 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 10:34:54.617871 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 10:34:54.617871 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:34:54.617871 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:34:54.617871 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:34:54.617871 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:34:54.617871 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:34:54.617871 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:34:54.617871 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:34:54.617871 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application with Lock( Jul 27 10:34:54.617871 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:34:54.617871 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:34:54.617871 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:34:54.617871 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:34:54.617871 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:34:54.617871 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:34:54.617871 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:34:54.617871 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:34:54.618835 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 10:34:54.618835 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 10:34:54.618835 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 10:34:54.618835 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Jul 27 10:34:54.625666 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 254/492] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 10:34:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 10:34:54.626833 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edde9252-38fd-465a-a669-07e62fdbd83e None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:54.657562 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:54.668210 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf87712-344b-4a28-9fe5-a8cd7f35b436 None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:54.703875 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce551c9-9822-4f5b-a67c-a95875ed679a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:54.704514 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce551c9-9822-4f5b-a67c-a95875ed679a None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:54.704795 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce551c9-9822-4f5b-a67c-a95875ed679a None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:54.705070 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce551c9-9822-4f5b-a67c-a95875ed679a None admin] PATH_INFO: `/v3/roles/service` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:54.716738 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce551c9-9822-4f5b-a67c-a95875ed679a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 10:34:54.723447 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application [None req-4ce551c9-9822-4f5b-a67c-a95875ed679a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 27 10:34:54.723447 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:34:54.723447 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:34:54.723447 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:34:54.723447 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:34:54.723447 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:34:54.723447 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:34:54.723447 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:34:54.723447 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:34:54.723447 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:34:54.723447 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:34:54.723447 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:34:54.723447 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 10:34:54.723447 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 10:34:54.723447 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 10:34:54.723447 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 10:34:54.724721 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 10:34:54.724721 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 10:34:54.724721 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 10:34:54.724721 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 10:34:54.724721 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:34:54.724721 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:34:54.724721 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:34:54.724721 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:34:54.724721 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:34:54.724721 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:34:54.724721 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:34:54.724721 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application with Lock( Jul 27 10:34:54.724721 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:34:54.724721 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:34:54.724721 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:34:54.724721 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:34:54.725657 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:34:54.725657 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:34:54.725657 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:34:54.725657 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:34:54.725657 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 10:34:54.725657 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 10:34:54.725657 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 10:34:54.725657 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 10:34:54.725657 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 10:34:54.725657 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 10:34:54.725657 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 27 10:34:54.725657 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Jul 27 10:34:54.725657 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 239/493] 10.222.0.36 () {56 vars in 1102 bytes} [Wed Jul 27 10:34:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 10:34:54.725657 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce551c9-9822-4f5b-a67c-a95875ed679a None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:54.743489 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edde9252-38fd-465a-a669-07e62fdbd83e None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:54.780362 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eaaa2c1-0d2a-4369-b2dd-51090fa45175 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:54.781025 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaaa2c1-0d2a-4369-b2dd-51090fa45175 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:54.781308 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaaa2c1-0d2a-4369-b2dd-51090fa45175 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:54.781583 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaaa2c1-0d2a-4369-b2dd-51090fa45175 None admin] PATH_INFO: `/v3/domains` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:54.782335 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eaaa2c1-0d2a-4369-b2dd-51090fa45175 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86827) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:34:54.782674 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eaaa2c1-0d2a-4369-b2dd-51090fa45175 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:54.784185 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eaaa2c1-0d2a-4369-b2dd-51090fa45175 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:54.802298 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 255/494] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 10:34:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:34:54.803778 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b99dac6-5ba0-4d9a-a079-65cdc49dcaf0 None None] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:54.804141 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b99dac6-5ba0-4d9a-a079-65cdc49dcaf0 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:54.804419 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b99dac6-5ba0-4d9a-a079-65cdc49dcaf0 None None] PATH_INFO: `/` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:54.805687 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 256/495] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 10:34:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 10:34:54.806379 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b99dac6-5ba0-4d9a-a079-65cdc49dcaf0 None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:54.836196 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:54.836591 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce551c9-9822-4f5b-a67c-a95875ed679a None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:54.871663 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23be0a87-3fed-474b-8fb7-b4f4107aaad9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:54.872271 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23be0a87-3fed-474b-8fb7-b4f4107aaad9 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:54.872548 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23be0a87-3fed-474b-8fb7-b4f4107aaad9 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:54.872820 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23be0a87-3fed-474b-8fb7-b4f4107aaad9 None admin] PATH_INFO: `/v3/domains` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:54.873506 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23be0a87-3fed-474b-8fb7-b4f4107aaad9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86828) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:34:54.873811 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23be0a87-3fed-474b-8fb7-b4f4107aaad9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:54.875229 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23be0a87-3fed-474b-8fb7-b4f4107aaad9 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:54.888020 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 240/496] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 10:34:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:34:54.888800 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23be0a87-3fed-474b-8fb7-b4f4107aaad9 None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:54.905553 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:54.926559 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b99dac6-5ba0-4d9a-a079-65cdc49dcaf0 None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:54.972092 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8f2908-81ee-4f9d-b48b-82b068b7e7fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:54.972716 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8f2908-81ee-4f9d-b48b-82b068b7e7fc None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:54.973002 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8f2908-81ee-4f9d-b48b-82b068b7e7fc None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:54.973277 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8f2908-81ee-4f9d-b48b-82b068b7e7fc None admin] PATH_INFO: `/v3/roles` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:54.973993 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8f2908-81ee-4f9d-b48b-82b068b7e7fc None admin] RBAC: Adding query filter params (name=service) {{(pid=86827) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:34:54.974300 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8f2908-81ee-4f9d-b48b-82b068b7e7fc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:54.975808 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8f2908-81ee-4f9d-b48b-82b068b7e7fc None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:54.985131 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 257/497] 10.222.0.36 () {56 vars in 1103 bytes} [Wed Jul 27 10:34:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:34:54.986369 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ff249a-027f-4ac7-8369-1574940d9fb7 None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:54.986693 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ff249a-027f-4ac7-8369-1574940d9fb7 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:54.987040 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ff249a-027f-4ac7-8369-1574940d9fb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:55.004777 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-60ff249a-027f-4ac7-8369-1574940d9fb7 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:34:55.005975 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-60ff249a-027f-4ac7-8369-1574940d9fb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:34:55.006535 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23be0a87-3fed-474b-8fb7-b4f4107aaad9 None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:55.041051 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ca5435-bb23-43d7-ab63-9ecfd4f67417 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:55.041685 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ca5435-bb23-43d7-ab63-9ecfd4f67417 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:55.041976 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ca5435-bb23-43d7-ab63-9ecfd4f67417 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:55.042250 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ca5435-bb23-43d7-ab63-9ecfd4f67417 None admin] PATH_INFO: `/v3/users/nova` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:55.049133 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ca5435-bb23-43d7-ab63-9ecfd4f67417 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:55.050576 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ca5435-bb23-43d7-ab63-9ecfd4f67417 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:55.063887 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application [None req-66ca5435-bb23-43d7-ab63-9ecfd4f67417 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 27 10:34:55.063887 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:34:55.063887 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:34:55.063887 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:34:55.063887 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:34:55.063887 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:34:55.063887 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:34:55.063887 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:34:55.063887 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:34:55.063887 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:34:55.063887 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:34:55.063887 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:34:55.063887 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 10:34:55.063887 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 10:34:55.063887 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 10:34:55.063887 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 10:34:55.064838 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:34:55.064838 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:34:55.064838 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 10:34:55.064838 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 10:34:55.064838 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 10:34:55.064838 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 10:34:55.064838 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:34:55.064838 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:34:55.064838 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:34:55.064838 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:34:55.064838 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:34:55.064838 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application with Lock( Jul 27 10:34:55.064838 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:34:55.064838 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:34:55.064838 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:34:55.064838 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:34:55.064838 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:34:55.064838 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:34:55.065804 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:34:55.065804 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:34:55.065804 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 10:34:55.065804 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 10:34:55.065804 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 10:34:55.065804 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 10:34:55.065804 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 10:34:55.065804 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 10:34:55.065804 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 27 10:34:55.065804 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Jul 27 10:34:55.065804 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 241/498] 10.222.0.36 () {56 vars in 1093 bytes} [Wed Jul 27 10:34:54 2022] GET /identity/v3/users/nova => generated 82 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 10:34:55.065804 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ca5435-bb23-43d7-ab63-9ecfd4f67417 None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:55.074355 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 258/499] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 10:34:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 10:34:55.075227 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ff249a-027f-4ac7-8369-1574940d9fb7 None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:55.092276 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:55.147403 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:55.175464 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ff249a-027f-4ac7-8369-1574940d9fb7 None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:55.177030 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ca5435-bb23-43d7-ab63-9ecfd4f67417 None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:55.209797 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b64247-6e8d-4257-bea0-1204f3601de4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:55.211052 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b64247-6e8d-4257-bea0-1204f3601de4 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:55.211245 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77f3d8a-6cb8-4ec5-ae84-d897517d1fc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:55.211349 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b64247-6e8d-4257-bea0-1204f3601de4 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:55.211596 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b64247-6e8d-4257-bea0-1204f3601de4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:55.211880 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77f3d8a-6cb8-4ec5-ae84-d897517d1fc9 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:55.212163 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77f3d8a-6cb8-4ec5-ae84-d897517d1fc9 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:55.212440 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77f3d8a-6cb8-4ec5-ae84-d897517d1fc9 None admin] PATH_INFO: `/v3/users/glance` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:55.219726 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77f3d8a-6cb8-4ec5-ae84-d897517d1fc9 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:55.221194 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77f3d8a-6cb8-4ec5-ae84-d897517d1fc9 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:55.221464 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b64247-6e8d-4257-bea0-1204f3601de4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:55.222937 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b64247-6e8d-4257-bea0-1204f3601de4 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:55.232333 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application [None req-b77f3d8a-6cb8-4ec5-ae84-d897517d1fc9 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 27 10:34:55.232333 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:34:55.232333 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:34:55.232333 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:34:55.232333 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:34:55.232333 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:34:55.232333 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:34:55.232333 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:34:55.232333 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:34:55.232333 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:34:55.232333 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:34:55.232333 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:34:55.232333 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 10:34:55.232333 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 10:34:55.232333 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 10:34:55.232333 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 10:34:55.233200 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:34:55.233200 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:34:55.233200 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 10:34:55.233200 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 10:34:55.233200 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 10:34:55.233200 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 10:34:55.233200 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:34:55.233200 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:34:55.233200 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:34:55.233200 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:34:55.233200 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:34:55.233200 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application with Lock( Jul 27 10:34:55.233200 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:34:55.233200 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:34:55.233200 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:34:55.233200 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:34:55.233200 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:34:55.233200 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:34:55.234087 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:34:55.234087 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:34:55.234087 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 10:34:55.234087 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 10:34:55.234087 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 10:34:55.234087 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 10:34:55.234087 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 10:34:55.234087 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 10:34:55.234087 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 27 10:34:55.234087 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Jul 27 10:34:55.234087 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 242/500] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jul 27 10:34:55 2022] GET /identity/v3/users/glance => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 10:34:55.235161 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2832e0d9-3b72-411a-9b6d-952902811e4f None None] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:55.235161 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2832e0d9-3b72-411a-9b6d-952902811e4f None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:55.235161 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2832e0d9-3b72-411a-9b6d-952902811e4f None None] PATH_INFO: `/` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:55.236547 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 243/501] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 10:34:55 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 10:34:55.237662 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application [None req-57b64247-6e8d-4257-bea0-1204f3601de4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 10:34:55.237662 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:34:55.237662 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:34:55.237662 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:34:55.237662 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:34:55.237662 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:34:55.237662 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:34:55.237662 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:34:55.237662 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:34:55.237662 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:34:55.237662 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:34:55.237662 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:34:55.237662 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 10:34:55.237662 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 10:34:55.237662 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 10:34:55.238774 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 10:34:55.238774 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:34:55.238774 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:34:55.238774 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:34:55.238774 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:34:55.238774 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:34:55.238774 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:34:55.238774 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:34:55.238774 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application with Lock( Jul 27 10:34:55.238774 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:34:55.238774 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:34:55.238774 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:34:55.238774 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:34:55.238774 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:34:55.238774 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:34:55.238774 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:34:55.238774 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:34:55.240332 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 10:34:55.240332 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 10:34:55.240332 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 10:34:55.240332 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Jul 27 10:34:55.240332 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 259/502] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 10:34:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 10:34:55.240944 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2832e0d9-3b72-411a-9b6d-952902811e4f None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:55.241191 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b64247-6e8d-4257-bea0-1204f3601de4 None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:55.307801 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:55.342131 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b64247-6e8d-4257-bea0-1204f3601de4 None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:55.342824 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2832e0d9-3b72-411a-9b6d-952902811e4f None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:55.370927 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:55.374858 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa905a9-74dd-4e56-b1f3-4ce79e2cfc31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:55.375486 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa905a9-74dd-4e56-b1f3-4ce79e2cfc31 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:55.375784 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa905a9-74dd-4e56-b1f3-4ce79e2cfc31 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:55.376050 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa905a9-74dd-4e56-b1f3-4ce79e2cfc31 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:55.376514 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5622d83-8cbe-457f-ad0b-d8e527b90523 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:55.377128 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5622d83-8cbe-457f-ad0b-d8e527b90523 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:55.377413 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5622d83-8cbe-457f-ad0b-d8e527b90523 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:55.377692 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5622d83-8cbe-457f-ad0b-d8e527b90523 None admin] PATH_INFO: `/v3/users` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:55.378493 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5622d83-8cbe-457f-ad0b-d8e527b90523 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86828) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:34:55.378837 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5622d83-8cbe-457f-ad0b-d8e527b90523 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:55.380279 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5622d83-8cbe-457f-ad0b-d8e527b90523 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:55.387028 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa905a9-74dd-4e56-b1f3-4ce79e2cfc31 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 10:34:55.392576 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application [None req-baa905a9-74dd-4e56-b1f3-4ce79e2cfc31 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 27 10:34:55.392576 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:34:55.392576 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:34:55.392576 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:34:55.392576 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:34:55.392576 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:34:55.392576 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:34:55.392576 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:34:55.392576 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:34:55.392576 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:34:55.392576 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:34:55.392576 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:34:55.392576 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 10:34:55.392576 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 10:34:55.392576 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 10:34:55.392576 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 10:34:55.393618 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 10:34:55.393618 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 10:34:55.393618 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 10:34:55.393618 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 10:34:55.393618 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:34:55.393618 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:34:55.393618 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:34:55.393618 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:34:55.393618 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:34:55.393618 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:34:55.393618 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:34:55.393618 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application with Lock( Jul 27 10:34:55.393618 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:34:55.393618 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:34:55.393618 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:34:55.393618 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:34:55.394413 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:34:55.394413 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:34:55.394413 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:34:55.394413 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:34:55.394413 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 10:34:55.394413 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 10:34:55.394413 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 10:34:55.394413 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 10:34:55.394413 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 10:34:55.394413 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 10:34:55.394413 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 27 10:34:55.394413 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Jul 27 10:34:55.394413 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 260/503] 10.222.0.36 () {56 vars in 1102 bytes} [Wed Jul 27 10:34:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 10:34:55.395470 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa905a9-74dd-4e56-b1f3-4ce79e2cfc31 None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:55.418254 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 244/504] 10.222.0.36 () {56 vars in 1133 bytes} [Wed Jul 27 10:34:55 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:34:55.419563 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c099c882-ec2d-4f59-a8e2-305c2c052931 None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:55.419839 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c099c882-ec2d-4f59-a8e2-305c2c052931 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:55.420122 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c099c882-ec2d-4f59-a8e2-305c2c052931 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:55.437990 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-c099c882-ec2d-4f59-a8e2-305c2c052931 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:34:55.439143 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-c099c882-ec2d-4f59-a8e2-305c2c052931 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:34:55.499524 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa905a9-74dd-4e56-b1f3-4ce79e2cfc31 None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:55.504209 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 245/505] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 10:34:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 10:34:55.505034 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c099c882-ec2d-4f59-a8e2-305c2c052931 None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:55.526335 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:55.534815 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca36146e-5ff4-4b80-ac30-05642009fe0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:55.535428 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca36146e-5ff4-4b80-ac30-05642009fe0e None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:55.535706 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca36146e-5ff4-4b80-ac30-05642009fe0e None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:55.535978 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca36146e-5ff4-4b80-ac30-05642009fe0e None admin] PATH_INFO: `/v3/users` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:55.536827 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca36146e-5ff4-4b80-ac30-05642009fe0e None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=86827) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:34:55.537079 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca36146e-5ff4-4b80-ac30-05642009fe0e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:55.538506 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca36146e-5ff4-4b80-ac30-05642009fe0e None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:55.574698 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 261/506] 10.222.0.36 () {56 vars in 1137 bytes} [Wed Jul 27 10:34:55 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:34:55.575706 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca36146e-5ff4-4b80-ac30-05642009fe0e None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:55.613268 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c099c882-ec2d-4f59-a8e2-305c2c052931 None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:55.633831 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:55.648299 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd4ec97-aba1-475d-8f4e-263d75ab1123 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:55.648910 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd4ec97-aba1-475d-8f4e-263d75ab1123 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:55.649191 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd4ec97-aba1-475d-8f4e-263d75ab1123 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:55.649467 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd4ec97-aba1-475d-8f4e-263d75ab1123 None admin] PATH_INFO: `/v3/domains` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:55.650156 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd4ec97-aba1-475d-8f4e-263d75ab1123 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86828) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:34:55.650456 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd4ec97-aba1-475d-8f4e-263d75ab1123 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:55.651881 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd4ec97-aba1-475d-8f4e-263d75ab1123 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:55.662763 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 246/507] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 10:34:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:34:55.663636 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd4ec97-aba1-475d-8f4e-263d75ab1123 None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:55.674131 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca36146e-5ff4-4b80-ac30-05642009fe0e None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:55.709368 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab939686-5dc5-4377-ada7-06a38e1794cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:55.709917 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab939686-5dc5-4377-ada7-06a38e1794cc None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:55.710372 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab939686-5dc5-4377-ada7-06a38e1794cc None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:55.710740 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab939686-5dc5-4377-ada7-06a38e1794cc None admin] PATH_INFO: `/v3/roles` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:55.711504 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab939686-5dc5-4377-ada7-06a38e1794cc None admin] RBAC: Adding query filter params (name=service) {{(pid=86827) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:34:55.711764 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab939686-5dc5-4377-ada7-06a38e1794cc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:55.713152 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab939686-5dc5-4377-ada7-06a38e1794cc None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:55.721542 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 262/508] 10.222.0.36 () {56 vars in 1103 bytes} [Wed Jul 27 10:34:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:34:55.722323 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab939686-5dc5-4377-ada7-06a38e1794cc None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:55.766027 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd4ec97-aba1-475d-8f4e-263d75ab1123 None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:55.773813 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:55.800192 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78001bd9-f84b-4555-b897-e67b6f191458 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:55.800813 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78001bd9-f84b-4555-b897-e67b6f191458 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:55.801097 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78001bd9-f84b-4555-b897-e67b6f191458 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:55.801781 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78001bd9-f84b-4555-b897-e67b6f191458 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:55.825971 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab939686-5dc5-4377-ada7-06a38e1794cc None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:55.848935 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:55.859616 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b00a8dc-8650-4d5a-88d1-6a219a30a610 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:55.860248 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b00a8dc-8650-4d5a-88d1-6a219a30a610 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:55.860526 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b00a8dc-8650-4d5a-88d1-6a219a30a610 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:55.860798 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b00a8dc-8650-4d5a-88d1-6a219a30a610 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:55.869500 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78001bd9-f84b-4555-b897-e67b6f191458 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:55.870993 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78001bd9-f84b-4555-b897-e67b6f191458 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:55.871673 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b00a8dc-8650-4d5a-88d1-6a219a30a610 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:55.873102 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b00a8dc-8650-4d5a-88d1-6a219a30a610 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:55.887627 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application [None req-8b00a8dc-8650-4d5a-88d1-6a219a30a610 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 10:34:55.887627 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:34:55.887627 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:34:55.887627 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:34:55.887627 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:34:55.887627 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:34:55.887627 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:34:55.887627 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:34:55.887627 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:34:55.887627 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:34:55.887627 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:34:55.887627 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:34:55.887627 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 10:34:55.887627 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 10:34:55.887627 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 10:34:55.888485 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 10:34:55.888485 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:34:55.888485 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:34:55.888485 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:34:55.888485 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:34:55.888485 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:34:55.888485 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:34:55.888485 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:34:55.888485 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application with Lock( Jul 27 10:34:55.888485 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:34:55.888485 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:34:55.888485 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:34:55.888485 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:34:55.888485 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:34:55.888485 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:34:55.888485 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:34:55.888485 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:34:55.889328 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 10:34:55.889328 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 10:34:55.889328 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 10:34:55.889328 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Jul 27 10:34:55.889328 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 263/509] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 10:34:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 10:34:55.889597 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b00a8dc-8650-4d5a-88d1-6a219a30a610 None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:55.971602 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 247/510] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jul 27 10:34:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 10:34:55.972368 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78001bd9-f84b-4555-b897-e67b6f191458 None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:55.992978 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:55.996078 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b00a8dc-8650-4d5a-88d1-6a219a30a610 None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:56.029431 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8223aa6a-d3ed-496f-a2a4-57ac8c0b8b87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:56.030051 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8223aa6a-d3ed-496f-a2a4-57ac8c0b8b87 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:56.030337 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8223aa6a-d3ed-496f-a2a4-57ac8c0b8b87 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:56.030634 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8223aa6a-d3ed-496f-a2a4-57ac8c0b8b87 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:56.055719 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:56.076059 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78001bd9-f84b-4555-b897-e67b6f191458 None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:56.101570 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8223aa6a-d3ed-496f-a2a4-57ac8c0b8b87 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:56.103080 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8223aa6a-d3ed-496f-a2a4-57ac8c0b8b87 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:56.110593 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920ac549-b5b2-484e-8b7a-576210a72413 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:56.111254 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920ac549-b5b2-484e-8b7a-576210a72413 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:56.111538 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920ac549-b5b2-484e-8b7a-576210a72413 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:56.111811 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920ac549-b5b2-484e-8b7a-576210a72413 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:56.118677 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920ac549-b5b2-484e-8b7a-576210a72413 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:56.120128 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920ac549-b5b2-484e-8b7a-576210a72413 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:56.131512 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application [None req-920ac549-b5b2-484e-8b7a-576210a72413 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 27 10:34:56.131512 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:34:56.131512 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:34:56.131512 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:34:56.131512 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:34:56.131512 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:34:56.131512 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:34:56.131512 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:34:56.131512 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:34:56.131512 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:34:56.131512 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:34:56.131512 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:34:56.131512 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 10:34:56.131512 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 10:34:56.131512 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 10:34:56.131512 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 10:34:56.132390 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:34:56.132390 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:34:56.132390 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 10:34:56.132390 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 10:34:56.132390 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 10:34:56.132390 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 10:34:56.132390 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:34:56.132390 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:34:56.132390 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:34:56.132390 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:34:56.132390 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:34:56.132390 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application with Lock( Jul 27 10:34:56.132390 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:34:56.132390 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:34:56.132390 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:34:56.132390 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:34:56.132390 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:34:56.132390 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:34:56.133275 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:34:56.133275 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:34:56.133275 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 10:34:56.133275 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 10:34:56.133275 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 10:34:56.133275 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 10:34:56.133275 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 10:34:56.133275 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 10:34:56.133275 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 27 10:34:56.133275 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Jul 27 10:34:56.133275 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 248/511] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jul 27 10:34:55 2022] GET /identity/v3/users/cinder => generated 84 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 10:34:56.133826 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920ac549-b5b2-484e-8b7a-576210a72413 None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:56.200742 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:56.204315 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 264/512] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jul 27 10:34:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 10:34:56.205273 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8223aa6a-d3ed-496f-a2a4-57ac8c0b8b87 None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:56.238530 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920ac549-b5b2-484e-8b7a-576210a72413 None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:56.270333 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec751b79-1211-4012-860e-1077726258d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:56.270987 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec751b79-1211-4012-860e-1077726258d0 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:56.271269 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec751b79-1211-4012-860e-1077726258d0 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:56.271539 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec751b79-1211-4012-860e-1077726258d0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:56.281239 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:56.283378 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec751b79-1211-4012-860e-1077726258d0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:56.284951 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec751b79-1211-4012-860e-1077726258d0 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:56.300150 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application [None req-ec751b79-1211-4012-860e-1077726258d0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 10:34:56.300150 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:34:56.300150 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:34:56.300150 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:34:56.300150 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:34:56.300150 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:34:56.300150 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:34:56.300150 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:34:56.300150 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:34:56.300150 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:34:56.300150 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:34:56.300150 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:34:56.300150 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 10:34:56.300150 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 10:34:56.300150 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 10:34:56.300995 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 10:34:56.300995 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:34:56.300995 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:34:56.300995 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:34:56.300995 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:34:56.300995 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:34:56.300995 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:34:56.300995 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:34:56.300995 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application with Lock( Jul 27 10:34:56.300995 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:34:56.300995 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:34:56.300995 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:34:56.300995 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:34:56.300995 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:34:56.300995 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:34:56.300995 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:34:56.300995 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:34:56.301820 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 10:34:56.301820 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 10:34:56.301820 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 10:34:56.301820 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Jul 27 10:34:56.301820 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 249/513] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 10:34:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 10:34:56.302089 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec751b79-1211-4012-860e-1077726258d0 None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:56.308027 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8223aa6a-d3ed-496f-a2a4-57ac8c0b8b87 None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:56.341724 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7aad01-051e-4124-9c6e-1f6873b7b15c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:56.342345 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7aad01-051e-4124-9c6e-1f6873b7b15c None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:56.342659 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7aad01-051e-4124-9c6e-1f6873b7b15c None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:56.342946 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7aad01-051e-4124-9c6e-1f6873b7b15c None admin] PATH_INFO: `/v3/domains` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:56.343643 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7aad01-051e-4124-9c6e-1f6873b7b15c None admin] RBAC: Adding query filter params (name=Default) {{(pid=86827) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:34:56.343952 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7aad01-051e-4124-9c6e-1f6873b7b15c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:56.345342 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7aad01-051e-4124-9c6e-1f6873b7b15c None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:56.356945 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 265/514] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 10:34:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:34:56.357733 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7aad01-051e-4124-9c6e-1f6873b7b15c None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:56.405318 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec751b79-1211-4012-860e-1077726258d0 None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:56.414877 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:56.438893 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fae210e-b654-4926-b550-ff04a463f368 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:56.439536 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fae210e-b654-4926-b550-ff04a463f368 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:56.439826 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fae210e-b654-4926-b550-ff04a463f368 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:56.440101 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fae210e-b654-4926-b550-ff04a463f368 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:56.451168 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fae210e-b654-4926-b550-ff04a463f368 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:56.452645 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fae210e-b654-4926-b550-ff04a463f368 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:56.454988 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7aad01-051e-4124-9c6e-1f6873b7b15c None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:56.467434 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application [None req-3fae210e-b654-4926-b550-ff04a463f368 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 10:34:56.467434 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:34:56.467434 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:34:56.467434 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:34:56.467434 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:34:56.467434 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:34:56.467434 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:34:56.467434 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:34:56.467434 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:34:56.467434 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:34:56.467434 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:34:56.467434 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:34:56.467434 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 10:34:56.467434 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 10:34:56.467434 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 10:34:56.468461 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 10:34:56.468461 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:34:56.468461 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:34:56.468461 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:34:56.468461 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:34:56.468461 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:34:56.468461 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:34:56.468461 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:34:56.468461 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application with Lock( Jul 27 10:34:56.468461 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:34:56.468461 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:34:56.468461 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:34:56.468461 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:34:56.468461 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:34:56.468461 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:34:56.468461 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:34:56.468461 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:34:56.469514 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 10:34:56.469514 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 10:34:56.469514 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 10:34:56.469514 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Jul 27 10:34:56.469514 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 250/515] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 10:34:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 10:34:56.469514 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fae210e-b654-4926-b550-ff04a463f368 None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:56.490901 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec0c6b4-0515-4f1f-97cc-505390e2b2f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:56.491452 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec0c6b4-0515-4f1f-97cc-505390e2b2f4 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:56.491734 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec0c6b4-0515-4f1f-97cc-505390e2b2f4 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:56.492011 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec0c6b4-0515-4f1f-97cc-505390e2b2f4 None admin] PATH_INFO: `/v3/users` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:56.492807 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec0c6b4-0515-4f1f-97cc-505390e2b2f4 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=86827) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:34:56.493120 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec0c6b4-0515-4f1f-97cc-505390e2b2f4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:56.494550 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec0c6b4-0515-4f1f-97cc-505390e2b2f4 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:56.511994 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 266/516] 10.222.0.36 () {56 vars in 1137 bytes} [Wed Jul 27 10:34:56 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:34:56.512785 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec0c6b4-0515-4f1f-97cc-505390e2b2f4 None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:56.520713 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:56.577606 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fae210e-b654-4926-b550-ff04a463f368 None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:56.610675 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4dd694-f0db-4bdd-bbce-072382d5f30f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:56.611312 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4dd694-f0db-4bdd-bbce-072382d5f30f None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:56.611434 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec0c6b4-0515-4f1f-97cc-505390e2b2f4 None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:56.611614 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4dd694-f0db-4bdd-bbce-072382d5f30f None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:56.611863 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4dd694-f0db-4bdd-bbce-072382d5f30f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:56.622041 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4dd694-f0db-4bdd-bbce-072382d5f30f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:56.623517 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4dd694-f0db-4bdd-bbce-072382d5f30f None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:56.628263 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:56.640110 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application [None req-8c4dd694-f0db-4bdd-bbce-072382d5f30f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 10:34:56.640110 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:34:56.640110 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:34:56.640110 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:34:56.640110 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:34:56.640110 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:34:56.640110 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:34:56.640110 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:34:56.640110 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:34:56.640110 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:34:56.640110 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:34:56.640110 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:34:56.640110 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 10:34:56.640110 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 10:34:56.640110 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 10:34:56.641159 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 10:34:56.641159 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:34:56.641159 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:34:56.641159 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:34:56.641159 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:34:56.641159 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:34:56.641159 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:34:56.641159 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:34:56.641159 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application with Lock( Jul 27 10:34:56.641159 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:34:56.641159 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:34:56.641159 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:34:56.641159 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:34:56.641159 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:34:56.641159 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:34:56.641159 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:34:56.641159 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:34:56.642132 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 10:34:56.642132 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 10:34:56.642132 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 10:34:56.642132 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Jul 27 10:34:56.642132 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 251/517] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 10:34:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 10:34:56.642132 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4dd694-f0db-4bdd-bbce-072382d5f30f None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:56.650724 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bc1e74-435e-4b78-9f81-3bcc27bd8cd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:56.651378 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bc1e74-435e-4b78-9f81-3bcc27bd8cd7 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:56.651666 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bc1e74-435e-4b78-9f81-3bcc27bd8cd7 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:56.651941 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bc1e74-435e-4b78-9f81-3bcc27bd8cd7 None admin] PATH_INFO: `/v3/domains` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:56.652640 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bc1e74-435e-4b78-9f81-3bcc27bd8cd7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86827) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:34:56.652951 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bc1e74-435e-4b78-9f81-3bcc27bd8cd7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:56.654337 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bc1e74-435e-4b78-9f81-3bcc27bd8cd7 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:56.665923 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 267/518] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 10:34:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:34:56.666755 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bc1e74-435e-4b78-9f81-3bcc27bd8cd7 None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:56.736050 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:56.743090 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4dd694-f0db-4bdd-bbce-072382d5f30f None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:56.769053 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bc1e74-435e-4b78-9f81-3bcc27bd8cd7 None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:56.780695 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042daafc-acb2-4bc4-8fd4-199ac1817964 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:56.781344 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042daafc-acb2-4bc4-8fd4-199ac1817964 None admin] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:56.781625 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042daafc-acb2-4bc4-8fd4-199ac1817964 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:56.781898 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042daafc-acb2-4bc4-8fd4-199ac1817964 None admin] PATH_INFO: `/v3/users` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:56.782582 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042daafc-acb2-4bc4-8fd4-199ac1817964 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:56.784064 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042daafc-acb2-4bc4-8fd4-199ac1817964 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:56.805044 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 252/519] 10.222.0.36 () {60 vars in 1132 bytes} [Wed Jul 27 10:34:56 2022] POST /identity/v3/users => generated 247 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 10:34:56.805852 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042daafc-acb2-4bc4-8fd4-199ac1817964 None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:56.808111 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fedbc1-1afb-4a03-ac23-99bd63848ee9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:56.809062 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fedbc1-1afb-4a03-ac23-99bd63848ee9 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:56.810152 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fedbc1-1afb-4a03-ac23-99bd63848ee9 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:56.810449 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fedbc1-1afb-4a03-ac23-99bd63848ee9 None admin] PATH_INFO: `/v3/domains` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:56.811222 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fedbc1-1afb-4a03-ac23-99bd63848ee9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86827) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:34:56.811539 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fedbc1-1afb-4a03-ac23-99bd63848ee9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:56.812989 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fedbc1-1afb-4a03-ac23-99bd63848ee9 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:56.835678 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 268/520] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 10:34:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:34:56.835678 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fedbc1-1afb-4a03-ac23-99bd63848ee9 None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:56.888538 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:56.921120 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042daafc-acb2-4bc4-8fd4-199ac1817964 None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:56.957842 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fedbc1-1afb-4a03-ac23-99bd63848ee9 None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:56.973613 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22c4e2e-9f2a-4e9e-a066-fd40830bad5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:56.974849 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22c4e2e-9f2a-4e9e-a066-fd40830bad5b None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:56.974849 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22c4e2e-9f2a-4e9e-a066-fd40830bad5b None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:56.974849 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22c4e2e-9f2a-4e9e-a066-fd40830bad5b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:57.000657 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:57.004628 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a35858-dfad-4e92-badd-acdd54397f9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:57.005775 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a35858-dfad-4e92-badd-acdd54397f9f None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:57.006424 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a35858-dfad-4e92-badd-acdd54397f9f None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:57.006873 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a35858-dfad-4e92-badd-acdd54397f9f None admin] PATH_INFO: `/v3/domains` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:57.007628 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a35858-dfad-4e92-badd-acdd54397f9f None admin] RBAC: Adding query filter params (name=Default) {{(pid=86827) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:34:57.007897 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a35858-dfad-4e92-badd-acdd54397f9f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:57.009547 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a35858-dfad-4e92-badd-acdd54397f9f None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:57.021620 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 269/521] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 10:34:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:34:57.023012 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a35858-dfad-4e92-badd-acdd54397f9f None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:57.055630 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22c4e2e-9f2a-4e9e-a066-fd40830bad5b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:57.057093 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22c4e2e-9f2a-4e9e-a066-fd40830bad5b None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:57.127297 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:57.132957 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a35858-dfad-4e92-badd-acdd54397f9f None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:57.169756 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 253/522] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jul 27 10:34:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 10:34:57.174090 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37980cd8-58d5-4871-8f4c-01ea55863766 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:57.174796 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22c4e2e-9f2a-4e9e-a066-fd40830bad5b None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:57.174904 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37980cd8-58d5-4871-8f4c-01ea55863766 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:57.175249 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37980cd8-58d5-4871-8f4c-01ea55863766 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:57.175532 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37980cd8-58d5-4871-8f4c-01ea55863766 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:57.183898 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37980cd8-58d5-4871-8f4c-01ea55863766 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:57.185454 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37980cd8-58d5-4871-8f4c-01ea55863766 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:57.198885 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application [None req-37980cd8-58d5-4871-8f4c-01ea55863766 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 27 10:34:57.198885 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:34:57.198885 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:34:57.198885 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:34:57.198885 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:34:57.198885 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:34:57.198885 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:34:57.198885 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:34:57.198885 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:34:57.198885 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:34:57.198885 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:34:57.198885 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:34:57.198885 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 10:34:57.198885 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 10:34:57.198885 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 10:34:57.198885 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 10:34:57.200001 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:34:57.200001 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:34:57.200001 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 10:34:57.200001 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 10:34:57.200001 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 10:34:57.200001 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 10:34:57.200001 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:34:57.200001 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:34:57.200001 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:34:57.200001 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:34:57.200001 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:34:57.200001 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application with Lock( Jul 27 10:34:57.200001 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:34:57.200001 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:34:57.200001 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:34:57.200001 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:34:57.200001 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:34:57.200001 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:34:57.200979 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:34:57.200979 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:34:57.200979 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 10:34:57.200979 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 10:34:57.200979 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 10:34:57.200979 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 10:34:57.200979 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 10:34:57.200979 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 10:34:57.200979 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 27 10:34:57.200979 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Jul 27 10:34:57.200979 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 270/523] 10.222.0.36 () {56 vars in 1102 bytes} [Wed Jul 27 10:34:57 2022] GET /identity/v3/users/neutron => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 10:34:57.201519 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37980cd8-58d5-4871-8f4c-01ea55863766 None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:57.236554 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:57.293537 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22c4e2e-9f2a-4e9e-a066-fd40830bad5b None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:57.312582 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37980cd8-58d5-4871-8f4c-01ea55863766 None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:57.329074 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6924aeee-c909-413d-9954-4ea180fe6ca7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:57.329921 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6924aeee-c909-413d-9954-4ea180fe6ca7 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:57.330372 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6924aeee-c909-413d-9954-4ea180fe6ca7 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:57.330883 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6924aeee-c909-413d-9954-4ea180fe6ca7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:57.338404 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6924aeee-c909-413d-9954-4ea180fe6ca7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:57.340021 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6924aeee-c909-413d-9954-4ea180fe6ca7 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:57.349676 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c07f3e6-2e1a-4455-bae8-34e5b0cf14d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:57.350322 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c07f3e6-2e1a-4455-bae8-34e5b0cf14d3 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:57.350649 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c07f3e6-2e1a-4455-bae8-34e5b0cf14d3 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:57.350941 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c07f3e6-2e1a-4455-bae8-34e5b0cf14d3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:57.353938 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application [None req-6924aeee-c909-413d-9954-4ea180fe6ca7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 10:34:57.353938 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:34:57.353938 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:34:57.353938 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:34:57.353938 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:34:57.353938 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:34:57.353938 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:34:57.353938 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:34:57.353938 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:34:57.353938 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:34:57.353938 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:34:57.353938 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:34:57.353938 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 10:34:57.353938 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 10:34:57.353938 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 10:34:57.355077 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 10:34:57.355077 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:34:57.355077 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:34:57.355077 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:34:57.355077 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:34:57.355077 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:34:57.355077 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:34:57.355077 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:34:57.355077 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application with Lock( Jul 27 10:34:57.355077 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:34:57.355077 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:34:57.355077 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:34:57.355077 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:34:57.355077 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:34:57.355077 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:34:57.355077 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:34:57.355077 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:34:57.356120 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 10:34:57.356120 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 10:34:57.356120 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 10:34:57.356120 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 10:34:57.356120 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 10:34:57.356120 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 10:34:57.356120 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 10:34:57.356120 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Jul 27 10:34:57.358638 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c07f3e6-2e1a-4455-bae8-34e5b0cf14d3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:57.359389 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c07f3e6-2e1a-4455-bae8-34e5b0cf14d3 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:57.362629 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 254/524] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Jul 27 10:34:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 10:34:57.363471 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6924aeee-c909-413d-9954-4ea180fe6ca7 None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:57.374370 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application [None req-4c07f3e6-2e1a-4455-bae8-34e5b0cf14d3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 10:34:57.374370 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:34:57.374370 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:34:57.374370 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:34:57.374370 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:34:57.374370 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:34:57.374370 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:34:57.374370 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:34:57.374370 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:34:57.374370 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:34:57.374370 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:34:57.374370 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:34:57.374370 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 10:34:57.374370 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 10:34:57.374370 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 10:34:57.375425 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 10:34:57.375425 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:34:57.375425 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:34:57.375425 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:34:57.375425 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:34:57.375425 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:34:57.375425 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:34:57.375425 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:34:57.375425 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application with Lock( Jul 27 10:34:57.375425 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:34:57.375425 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:34:57.375425 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:34:57.375425 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:34:57.375425 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:34:57.375425 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:34:57.375425 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:34:57.375425 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:34:57.381483 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 10:34:57.381483 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 10:34:57.381483 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 10:34:57.381483 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 10:34:57.381483 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 10:34:57.381483 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 10:34:57.381483 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 10:34:57.381483 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Jul 27 10:34:57.381483 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 271/525] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Jul 27 10:34:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 10:34:57.381483 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c07f3e6-2e1a-4455-bae8-34e5b0cf14d3 None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:57.400649 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:57.489141 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6924aeee-c909-413d-9954-4ea180fe6ca7 None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:57.495923 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c07f3e6-2e1a-4455-bae8-34e5b0cf14d3 None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:57.506389 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:57.531896 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e00d20-dbd0-42aa-b819-a66421f8b415 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:57.532550 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e00d20-dbd0-42aa-b819-a66421f8b415 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:57.532833 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e00d20-dbd0-42aa-b819-a66421f8b415 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:57.533109 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e00d20-dbd0-42aa-b819-a66421f8b415 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:57.534023 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9092fdc-a7ac-4b02-8593-8599a8f55411 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:57.534796 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9092fdc-a7ac-4b02-8593-8599a8f55411 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:57.535139 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9092fdc-a7ac-4b02-8593-8599a8f55411 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:57.535479 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9092fdc-a7ac-4b02-8593-8599a8f55411 None admin] PATH_INFO: `/v3/users` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:57.536442 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9092fdc-a7ac-4b02-8593-8599a8f55411 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=86827) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:34:57.536813 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9092fdc-a7ac-4b02-8593-8599a8f55411 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:57.539075 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9092fdc-a7ac-4b02-8593-8599a8f55411 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:57.545301 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e00d20-dbd0-42aa-b819-a66421f8b415 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:57.546716 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e00d20-dbd0-42aa-b819-a66421f8b415 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:57.569010 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 272/526] 10.222.0.36 () {56 vars in 1139 bytes} [Wed Jul 27 10:34:57 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:34:57.569010 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9092fdc-a7ac-4b02-8593-8599a8f55411 None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:57.569298 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application [None req-56e00d20-dbd0-42aa-b819-a66421f8b415 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 10:34:57.569298 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:34:57.569298 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:34:57.569298 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:34:57.569298 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:34:57.569298 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:34:57.569298 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:34:57.569298 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:34:57.569298 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:34:57.569298 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:34:57.569298 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:34:57.569298 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:34:57.569298 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 10:34:57.569298 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 10:34:57.569298 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 10:34:57.570287 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 10:34:57.570287 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:34:57.570287 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:34:57.570287 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:34:57.570287 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:34:57.570287 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:34:57.570287 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:34:57.570287 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:34:57.570287 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application with Lock( Jul 27 10:34:57.570287 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:34:57.570287 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:34:57.570287 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:34:57.570287 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:34:57.570287 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:34:57.570287 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:34:57.570287 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:34:57.570287 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:34:57.571380 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 10:34:57.571380 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 10:34:57.571380 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 10:34:57.571380 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Jul 27 10:34:57.571380 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 255/527] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 10:34:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 10:34:57.571380 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e00d20-dbd0-42aa-b819-a66421f8b415 None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:57.651605 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:57.676798 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9092fdc-a7ac-4b02-8593-8599a8f55411 None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:57.677118 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e00d20-dbd0-42aa-b819-a66421f8b415 None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:57.711455 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcf25a9-4182-40f5-a331-12e7afe8065b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:57.711623 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a200e9-d889-44b0-9a79-a29781d6d6de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:57.712068 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a200e9-d889-44b0-9a79-a29781d6d6de None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:57.712168 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcf25a9-4182-40f5-a331-12e7afe8065b None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:57.712350 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcf25a9-4182-40f5-a331-12e7afe8065b None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:57.712441 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a200e9-d889-44b0-9a79-a29781d6d6de None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:57.712620 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a200e9-d889-44b0-9a79-a29781d6d6de None admin] PATH_INFO: `/v3/projects` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:57.712722 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcf25a9-4182-40f5-a331-12e7afe8065b None admin] PATH_INFO: `/v3/projects` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:57.713367 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcf25a9-4182-40f5-a331-12e7afe8065b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86827) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:34:57.713467 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a200e9-d889-44b0-9a79-a29781d6d6de None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86828) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:34:57.713670 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a200e9-d889-44b0-9a79-a29781d6d6de None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:57.713761 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcf25a9-4182-40f5-a331-12e7afe8065b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:57.715147 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a200e9-d889-44b0-9a79-a29781d6d6de None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:57.715256 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcf25a9-4182-40f5-a331-12e7afe8065b None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:57.742847 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 273/528] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Jul 27 10:34:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:34:57.742847 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcf25a9-4182-40f5-a331-12e7afe8065b None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:57.743151 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 256/529] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Jul 27 10:34:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:34:57.743151 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a200e9-d889-44b0-9a79-a29781d6d6de None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:57.773164 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:57.848398 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcf25a9-4182-40f5-a331-12e7afe8065b None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:57.861565 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a200e9-d889-44b0-9a79-a29781d6d6de None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:57.885695 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae70b084-0494-4e63-bd5c-d68213e64fb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:57.886507 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae70b084-0494-4e63-bd5c-d68213e64fb3 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:57.886963 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae70b084-0494-4e63-bd5c-d68213e64fb3 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:57.887239 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae70b084-0494-4e63-bd5c-d68213e64fb3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:57.891531 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:57.898089 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c5082d-edea-462d-b8ab-cc95201ca7d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:57.898711 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c5082d-edea-462d-b8ab-cc95201ca7d3 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:57.899000 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c5082d-edea-462d-b8ab-cc95201ca7d3 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:57.899278 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c5082d-edea-462d-b8ab-cc95201ca7d3 None admin] PATH_INFO: `/v3/domains` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:57.900119 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c5082d-edea-462d-b8ab-cc95201ca7d3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86828) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:34:57.900350 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c5082d-edea-462d-b8ab-cc95201ca7d3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:57.901749 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c5082d-edea-462d-b8ab-cc95201ca7d3 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:57.914527 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 257/530] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 10:34:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:34:57.915371 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c5082d-edea-462d-b8ab-cc95201ca7d3 None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:57.976066 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae70b084-0494-4e63-bd5c-d68213e64fb3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:57.977836 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae70b084-0494-4e63-bd5c-d68213e64fb3 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:58.007334 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:58.025744 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c5082d-edea-462d-b8ab-cc95201ca7d3 None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:58.060815 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b194a3f-6598-4dc7-9537-153edebd1391 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:58.061429 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b194a3f-6598-4dc7-9537-153edebd1391 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:58.061707 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b194a3f-6598-4dc7-9537-153edebd1391 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:58.062180 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b194a3f-6598-4dc7-9537-153edebd1391 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:58.062789 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b194a3f-6598-4dc7-9537-153edebd1391 None admin] RBAC: Adding query filter params (role.id=9e0038c6bbd34556af3f84f9e2a7c2c5, scope.project.id=5f296d8e3b8f4001909addc49d8dac01, user.id=bc03c5d232654878aa1ddce15f49fdaa) {{(pid=86828) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:34:58.063097 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b194a3f-6598-4dc7-9537-153edebd1391 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:58.064527 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b194a3f-6598-4dc7-9537-153edebd1391 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:58.073130 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 258/531] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Jul 27 10:34:57 2022] GET /identity/v3/role_assignments?user.id=bc03c5d232654878aa1ddce15f49fdaa&scope.project.id=5f296d8e3b8f4001909addc49d8dac01&role.id=9e0038c6bbd34556af3f84f9e2a7c2c5 => generated 259 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:34:58.079951 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b194a3f-6598-4dc7-9537-153edebd1391 None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:58.090899 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 274/532] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jul 27 10:34:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 10:34:58.091795 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae70b084-0494-4e63-bd5c-d68213e64fb3 None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:58.132442 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:58.222704 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae70b084-0494-4e63-bd5c-d68213e64fb3 None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:58.225168 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b194a3f-6598-4dc7-9537-153edebd1391 None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:58.263064 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d777fb1-b8dc-4812-94d5-0269d88d80c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:58.263689 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d777fb1-b8dc-4812-94d5-0269d88d80c7 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:58.263970 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d777fb1-b8dc-4812-94d5-0269d88d80c7 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:58.264239 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d777fb1-b8dc-4812-94d5-0269d88d80c7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:58.265601 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d777fb1-b8dc-4812-94d5-0269d88d80c7 None admin] RBAC: Adding query filter params (role.id=9e0038c6bbd34556af3f84f9e2a7c2c5, scope.project.id=5f296d8e3b8f4001909addc49d8dac01, user.id=e059d8b6c28c4fcba925d05d7c98e4ac) {{(pid=86828) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:34:58.265919 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d777fb1-b8dc-4812-94d5-0269d88d80c7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:58.267392 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d777fb1-b8dc-4812-94d5-0269d88d80c7 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:58.270656 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47df3c74-f4c9-4703-85ea-4817aa8d539f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:58.271829 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47df3c74-f4c9-4703-85ea-4817aa8d539f None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:58.272350 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47df3c74-f4c9-4703-85ea-4817aa8d539f None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:58.273395 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47df3c74-f4c9-4703-85ea-4817aa8d539f None admin] PATH_INFO: `/v3/projects/service` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:58.278709 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 259/533] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Jul 27 10:34:58 2022] GET /identity/v3/role_assignments?user.id=e059d8b6c28c4fcba925d05d7c98e4ac&scope.project.id=5f296d8e3b8f4001909addc49d8dac01&role.id=9e0038c6bbd34556af3f84f9e2a7c2c5 => generated 259 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:34:58.280955 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d777fb1-b8dc-4812-94d5-0269d88d80c7 None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:58.291503 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47df3c74-f4c9-4703-85ea-4817aa8d539f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:58.296401 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47df3c74-f4c9-4703-85ea-4817aa8d539f None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:58.324314 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application [None req-47df3c74-f4c9-4703-85ea-4817aa8d539f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 10:34:58.324314 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:34:58.324314 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:34:58.324314 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:34:58.324314 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:34:58.324314 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:34:58.324314 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:34:58.324314 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:34:58.324314 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:34:58.324314 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:34:58.324314 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:34:58.324314 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:34:58.324314 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 10:34:58.324314 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 10:34:58.324314 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 10:34:58.325235 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 10:34:58.325235 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:34:58.325235 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:34:58.325235 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:34:58.325235 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:34:58.325235 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:34:58.325235 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:34:58.325235 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:34:58.325235 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application with Lock( Jul 27 10:34:58.325235 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:34:58.325235 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:34:58.325235 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:34:58.325235 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:34:58.325235 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:34:58.325235 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:34:58.325235 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:34:58.325235 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:34:58.326057 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 10:34:58.326057 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 10:34:58.326057 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 10:34:58.326057 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 10:34:58.326057 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 10:34:58.326057 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 10:34:58.326057 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 10:34:58.326057 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Jul 27 10:34:58.327039 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:58.333383 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 275/534] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Jul 27 10:34:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 10:34:58.334636 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47df3c74-f4c9-4703-85ea-4817aa8d539f None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:58.437015 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d777fb1-b8dc-4812-94d5-0269d88d80c7 None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:58.474973 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afda883-561f-4bda-bec1-0b204bae211c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:58.476059 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afda883-561f-4bda-bec1-0b204bae211c None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:58.476547 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afda883-561f-4bda-bec1-0b204bae211c None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:58.477053 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afda883-561f-4bda-bec1-0b204bae211c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:58.488692 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afda883-561f-4bda-bec1-0b204bae211c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:58.490462 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afda883-561f-4bda-bec1-0b204bae211c None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:58.506084 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application [None req-0afda883-561f-4bda-bec1-0b204bae211c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 10:34:58.506084 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:34:58.506084 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:34:58.506084 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:34:58.506084 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:34:58.506084 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:34:58.506084 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:34:58.506084 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:34:58.506084 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:34:58.506084 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:34:58.506084 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:34:58.506084 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:34:58.506084 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 10:34:58.506084 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 10:34:58.506084 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 10:34:58.507581 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 10:34:58.507581 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:34:58.507581 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:34:58.507581 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:34:58.507581 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:34:58.507581 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:34:58.507581 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:34:58.507581 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:34:58.507581 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application with Lock( Jul 27 10:34:58.507581 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:34:58.507581 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:34:58.507581 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:34:58.507581 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:34:58.507581 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:34:58.507581 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:34:58.507581 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:34:58.507581 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:34:58.508407 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 10:34:58.508407 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 10:34:58.508407 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 10:34:58.508407 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Jul 27 10:34:58.510905 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 260/535] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 10:34:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 10:34:58.519594 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afda883-561f-4bda-bec1-0b204bae211c None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:58.570908 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:58.580981 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47df3c74-f4c9-4703-85ea-4817aa8d539f None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:58.633479 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ca563b-1580-47fe-94ed-39aaef8867fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:58.634752 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ca563b-1580-47fe-94ed-39aaef8867fc None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:58.635041 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ca563b-1580-47fe-94ed-39aaef8867fc None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:58.635319 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ca563b-1580-47fe-94ed-39aaef8867fc None admin] PATH_INFO: `/v3/projects` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:58.636241 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ca563b-1580-47fe-94ed-39aaef8867fc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86827) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:34:58.636566 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ca563b-1580-47fe-94ed-39aaef8867fc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:58.638287 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ca563b-1580-47fe-94ed-39aaef8867fc None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:58.642985 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:58.654449 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 276/536] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Jul 27 10:34:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:34:58.663627 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ca563b-1580-47fe-94ed-39aaef8867fc None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:58.774360 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ca563b-1580-47fe-94ed-39aaef8867fc None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:58.780585 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afda883-561f-4bda-bec1-0b204bae211c None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:58.814707 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f940d7-5efd-4518-9025-204e515ddf18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:58.815337 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f940d7-5efd-4518-9025-204e515ddf18 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:58.815622 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f940d7-5efd-4518-9025-204e515ddf18 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:58.815899 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f940d7-5efd-4518-9025-204e515ddf18 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:58.817229 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f940d7-5efd-4518-9025-204e515ddf18 None admin] RBAC: Adding query filter params (role.id=9e0038c6bbd34556af3f84f9e2a7c2c5, scope.project.id=5f296d8e3b8f4001909addc49d8dac01, user.id=864d35f268cf45148756109759bb2743) {{(pid=86827) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:34:58.817546 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f940d7-5efd-4518-9025-204e515ddf18 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:58.819099 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f940d7-5efd-4518-9025-204e515ddf18 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:58.826058 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 277/537] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Jul 27 10:34:58 2022] GET /identity/v3/role_assignments?user.id=864d35f268cf45148756109759bb2743&scope.project.id=5f296d8e3b8f4001909addc49d8dac01&role.id=9e0038c6bbd34556af3f84f9e2a7c2c5 => generated 259 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:34:58.862316 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a747d86-9aab-4b1c-8e96-da5988e6cc9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:58.862992 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a747d86-9aab-4b1c-8e96-da5988e6cc9b None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:58.863279 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a747d86-9aab-4b1c-8e96-da5988e6cc9b None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:58.863561 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a747d86-9aab-4b1c-8e96-da5988e6cc9b None admin] PATH_INFO: `/v3/domains` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:58.864289 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a747d86-9aab-4b1c-8e96-da5988e6cc9b None admin] RBAC: Adding query filter params (name=Default) {{(pid=86828) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:34:58.864597 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a747d86-9aab-4b1c-8e96-da5988e6cc9b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:58.866025 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a747d86-9aab-4b1c-8e96-da5988e6cc9b None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:58.892630 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 261/538] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 10:34:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 373 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:34:58.899768 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f940d7-5efd-4518-9025-204e515ddf18 None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:58.900662 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a202c495-4699-4ae0-9206-8664428440f1 None None] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:58.901031 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a202c495-4699-4ae0-9206-8664428440f1 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:58.901312 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a202c495-4699-4ae0-9206-8664428440f1 None None] PATH_INFO: `/` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:58.902911 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 262/539] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 10:34:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 10:34:58.908496 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26637d57-bc18-48ad-a421-a422d5ce3ebc None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:58.908785 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26637d57-bc18-48ad-a421-a422d5ce3ebc None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:58.909062 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26637d57-bc18-48ad-a421-a422d5ce3ebc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:58.940515 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-26637d57-bc18-48ad-a421-a422d5ce3ebc None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:34:58.941779 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-26637d57-bc18-48ad-a421-a422d5ce3ebc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:34:58.953725 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:59.075212 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 263/540] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 10:34:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 27 10:34:59.116577 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f940d7-5efd-4518-9025-204e515ddf18 None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:59.128792 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26637d57-bc18-48ad-a421-a422d5ce3ebc None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:59.184680 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477d3636-2980-41f0-96cf-7bb2ae2af257 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:59.185706 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477d3636-2980-41f0-96cf-7bb2ae2af257 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:59.186227 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477d3636-2980-41f0-96cf-7bb2ae2af257 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:59.186732 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477d3636-2980-41f0-96cf-7bb2ae2af257 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:59.200848 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:59.202271 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477d3636-2980-41f0-96cf-7bb2ae2af257 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:59.204063 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477d3636-2980-41f0-96cf-7bb2ae2af257 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:59.228667 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application [None req-477d3636-2980-41f0-96cf-7bb2ae2af257 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 10:34:59.228667 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:34:59.228667 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:34:59.228667 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:34:59.228667 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:34:59.228667 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:34:59.228667 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:34:59.228667 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:34:59.228667 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:34:59.228667 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:34:59.228667 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:34:59.228667 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:34:59.228667 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 10:34:59.228667 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 10:34:59.228667 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 10:34:59.229570 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 10:34:59.229570 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:34:59.229570 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:34:59.229570 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:34:59.229570 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:34:59.229570 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:34:59.229570 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:34:59.229570 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:34:59.229570 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application with Lock( Jul 27 10:34:59.229570 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:34:59.229570 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:34:59.229570 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:34:59.229570 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:34:59.229570 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:34:59.229570 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:34:59.229570 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:34:59.229570 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:34:59.230402 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 10:34:59.230402 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 10:34:59.230402 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 10:34:59.230402 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 10:34:59.230402 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 10:34:59.230402 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 10:34:59.230402 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 10:34:59.230402 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Jul 27 10:34:59.233479 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 278/541] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Jul 27 10:34:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 335 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 10:34:59.241356 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477d3636-2980-41f0-96cf-7bb2ae2af257 None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:59.290864 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26637d57-bc18-48ad-a421-a422d5ce3ebc None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:59.352739 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4789db2e-a5d4-4667-8f72-b4a0158f03e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:59.353680 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4789db2e-a5d4-4667-8f72-b4a0158f03e3 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:59.354858 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4789db2e-a5d4-4667-8f72-b4a0158f03e3 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:59.355954 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4789db2e-a5d4-4667-8f72-b4a0158f03e3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:59.382477 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4789db2e-a5d4-4667-8f72-b4a0158f03e3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 10:34:59.392542 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application [None req-4789db2e-a5d4-4667-8f72-b4a0158f03e3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 27 10:34:59.392542 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:34:59.392542 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:34:59.392542 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:34:59.392542 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:34:59.392542 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:34:59.392542 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:34:59.392542 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:34:59.392542 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:34:59.392542 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:34:59.392542 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:34:59.392542 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:34:59.392542 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 10:34:59.392542 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 10:34:59.392542 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 10:34:59.392542 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 10:34:59.393491 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 10:34:59.393491 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 10:34:59.393491 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 10:34:59.393491 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 10:34:59.393491 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:34:59.393491 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:34:59.393491 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:34:59.393491 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:34:59.393491 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:34:59.393491 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:34:59.393491 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:34:59.393491 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application with Lock( Jul 27 10:34:59.393491 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:34:59.393491 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:34:59.393491 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:34:59.393491 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:34:59.394264 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:34:59.394264 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:34:59.394264 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:34:59.394264 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:34:59.394264 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 10:34:59.394264 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 10:34:59.394264 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 10:34:59.394264 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 10:34:59.394264 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 10:34:59.394264 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 10:34:59.394264 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 27 10:34:59.394264 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Jul 27 10:34:59.400737 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 264/542] 10.222.0.36 () {56 vars in 1102 bytes} [Wed Jul 27 10:34:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 10:34:59.406177 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4789db2e-a5d4-4667-8f72-b4a0158f03e3 None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:59.409976 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:59.433648 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477d3636-2980-41f0-96cf-7bb2ae2af257 None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:59.481574 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b2c565-0134-4b35-9ef8-0295b96b9702 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:59.482564 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b2c565-0134-4b35-9ef8-0295b96b9702 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:59.483154 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b2c565-0134-4b35-9ef8-0295b96b9702 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:59.483580 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b2c565-0134-4b35-9ef8-0295b96b9702 None admin] PATH_INFO: `/v3/projects` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:59.484498 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b2c565-0134-4b35-9ef8-0295b96b9702 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86827) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:34:59.485134 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b2c565-0134-4b35-9ef8-0295b96b9702 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:59.486873 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b2c565-0134-4b35-9ef8-0295b96b9702 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:59.512933 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 279/543] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Jul 27 10:34:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 10:34:59.520662 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b2c565-0134-4b35-9ef8-0295b96b9702 None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:59.546396 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4789db2e-a5d4-4667-8f72-b4a0158f03e3 None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:59.579464 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:34:59.621137 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fddb172-ea15-42af-a50b-4233b1132b4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:59.621991 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fddb172-ea15-42af-a50b-4233b1132b4b None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:59.622442 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fddb172-ea15-42af-a50b-4233b1132b4b None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:59.622888 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fddb172-ea15-42af-a50b-4233b1132b4b None admin] PATH_INFO: `/v3/roles` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:59.623782 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fddb172-ea15-42af-a50b-4233b1132b4b None admin] RBAC: Adding query filter params (name=service) {{(pid=86828) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:34:59.624234 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fddb172-ea15-42af-a50b-4233b1132b4b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:59.625752 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fddb172-ea15-42af-a50b-4233b1132b4b None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:59.645096 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 265/544] 10.222.0.36 () {56 vars in 1103 bytes} [Wed Jul 27 10:34:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:34:59.646163 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fddb172-ea15-42af-a50b-4233b1132b4b None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:59.691969 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b2c565-0134-4b35-9ef8-0295b96b9702 None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:59.758297 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fcb8a8-1fad-4854-918a-0102a954537a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:59.759528 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fcb8a8-1fad-4854-918a-0102a954537a None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:59.759961 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fcb8a8-1fad-4854-918a-0102a954537a None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:59.761348 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fcb8a8-1fad-4854-918a-0102a954537a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:59.762299 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fcb8a8-1fad-4854-918a-0102a954537a None admin] RBAC: Adding query filter params (role.id=9e0038c6bbd34556af3f84f9e2a7c2c5, scope.project.id=5f296d8e3b8f4001909addc49d8dac01, user.id=c7922d0a1a7c4e808d0b2a80dca0da40) {{(pid=86827) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:34:59.763750 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fcb8a8-1fad-4854-918a-0102a954537a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:59.765361 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fcb8a8-1fad-4854-918a-0102a954537a None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:59.778139 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 280/545] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Jul 27 10:34:59 2022] GET /identity/v3/role_assignments?user.id=c7922d0a1a7c4e808d0b2a80dca0da40&scope.project.id=5f296d8e3b8f4001909addc49d8dac01&role.id=9e0038c6bbd34556af3f84f9e2a7c2c5 => generated 259 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 10:34:59.804848 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fddb172-ea15-42af-a50b-4233b1132b4b None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:34:59.847997 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8b9e20-1ec6-463f-8199-1010a4581825 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:34:59.848952 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8b9e20-1ec6-463f-8199-1010a4581825 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:34:59.849401 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8b9e20-1ec6-463f-8199-1010a4581825 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:34:59.849825 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8b9e20-1ec6-463f-8199-1010a4581825 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:34:59.867267 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8b9e20-1ec6-463f-8199-1010a4581825 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:34:59.869017 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8b9e20-1ec6-463f-8199-1010a4581825 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:34:59.892164 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application [None req-ef8b9e20-1ec6-463f-8199-1010a4581825 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 10:34:59.892164 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:34:59.892164 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:34:59.892164 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:34:59.892164 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:34:59.892164 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:34:59.892164 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:34:59.892164 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:34:59.892164 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:34:59.892164 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:34:59.892164 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:34:59.892164 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:34:59.892164 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 10:34:59.892164 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 10:34:59.892164 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 10:34:59.893275 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 10:34:59.893275 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:34:59.893275 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:34:59.893275 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:34:59.893275 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:34:59.893275 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:34:59.893275 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:34:59.893275 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:34:59.893275 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application with Lock( Jul 27 10:34:59.893275 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:34:59.893275 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:34:59.893275 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:34:59.893275 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:34:59.893275 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:34:59.893275 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:34:59.893275 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:34:59.893275 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:34:59.894324 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 10:34:59.894324 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 10:34:59.894324 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 10:34:59.894324 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Jul 27 10:34:59.895977 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 266/546] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 10:34:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 10:34:59.901821 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fcb8a8-1fad-4854-918a-0102a954537a None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:34:59.907589 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:00.035587 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fcb8a8-1fad-4854-918a-0102a954537a None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:00.087655 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a2c4b7-c8b0-4ba3-bbf3-3c12f2360664 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:00.088883 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a2c4b7-c8b0-4ba3-bbf3-3c12f2360664 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:00.088883 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a2c4b7-c8b0-4ba3-bbf3-3c12f2360664 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:00.088883 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a2c4b7-c8b0-4ba3-bbf3-3c12f2360664 None admin] PATH_INFO: `/v3/domains` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:00.089574 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a2c4b7-c8b0-4ba3-bbf3-3c12f2360664 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86827) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:35:00.089880 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a2c4b7-c8b0-4ba3-bbf3-3c12f2360664 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:00.091301 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a2c4b7-c8b0-4ba3-bbf3-3c12f2360664 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:00.115891 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 281/547] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 10:34:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:35:00.122194 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8b9e20-1ec6-463f-8199-1010a4581825 None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:00.177849 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:00.275028 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8b9e20-1ec6-463f-8199-1010a4581825 None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:00.310509 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5752dc8b-9ec9-40eb-b4c0-00bc5947651c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:00.311259 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5752dc8b-9ec9-40eb-b4c0-00bc5947651c None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:00.311544 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5752dc8b-9ec9-40eb-b4c0-00bc5947651c None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:00.311820 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5752dc8b-9ec9-40eb-b4c0-00bc5947651c None admin] PATH_INFO: `/v3/users/swift` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:00.327759 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5752dc8b-9ec9-40eb-b4c0-00bc5947651c None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:00.329225 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5752dc8b-9ec9-40eb-b4c0-00bc5947651c None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:00.344223 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application [None req-5752dc8b-9ec9-40eb-b4c0-00bc5947651c None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 27 10:35:00.344223 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:35:00.344223 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:35:00.344223 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:35:00.344223 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:35:00.344223 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:35:00.344223 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:35:00.344223 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:35:00.344223 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:35:00.344223 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:35:00.344223 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:35:00.344223 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:35:00.344223 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 10:35:00.344223 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 10:35:00.344223 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 10:35:00.344223 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 10:35:00.345211 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:35:00.345211 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:35:00.345211 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 10:35:00.345211 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 10:35:00.345211 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 10:35:00.345211 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 10:35:00.345211 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:35:00.345211 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:35:00.345211 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:35:00.345211 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:35:00.345211 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:35:00.345211 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application with Lock( Jul 27 10:35:00.345211 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:35:00.345211 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:35:00.345211 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:35:00.345211 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:35:00.345211 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:35:00.345211 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:35:00.346587 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:35:00.346587 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:35:00.346587 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 10:35:00.346587 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 10:35:00.346587 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 10:35:00.346587 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 10:35:00.346587 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 10:35:00.346587 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 10:35:00.346587 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 27 10:35:00.346587 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Jul 27 10:35:00.346587 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 267/548] 10.222.0.36 () {56 vars in 1096 bytes} [Wed Jul 27 10:35:00 2022] GET /identity/v3/users/swift => generated 83 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 10:35:00.351602 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a2c4b7-c8b0-4ba3-bbf3-3c12f2360664 None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:00.471479 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:00.488491 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bd43d3-753b-4d1f-be5c-bc1facc00c1b None None] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:00.489193 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bd43d3-753b-4d1f-be5c-bc1facc00c1b None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:00.489480 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bd43d3-753b-4d1f-be5c-bc1facc00c1b None None] PATH_INFO: `/` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:00.491520 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 268/549] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 10:35:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 10:35:00.496766 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619a31cc-4645-4580-8714-819c56f3000f None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:00.497058 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619a31cc-4645-4580-8714-819c56f3000f None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:00.497513 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619a31cc-4645-4580-8714-819c56f3000f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:00.529932 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a2c4b7-c8b0-4ba3-bbf3-3c12f2360664 None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:00.532237 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-619a31cc-4645-4580-8714-819c56f3000f None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:35:00.533459 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-619a31cc-4645-4580-8714-819c56f3000f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:35:00.603451 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213e38f7-f5a8-450d-87b3-1aecd95d835f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:00.604087 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213e38f7-f5a8-450d-87b3-1aecd95d835f None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:00.604372 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213e38f7-f5a8-450d-87b3-1aecd95d835f None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:00.604653 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213e38f7-f5a8-450d-87b3-1aecd95d835f None admin] PATH_INFO: `/v3/users` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:00.605463 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213e38f7-f5a8-450d-87b3-1aecd95d835f None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=86827) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:35:00.605773 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213e38f7-f5a8-450d-87b3-1aecd95d835f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:00.607270 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213e38f7-f5a8-450d-87b3-1aecd95d835f None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:00.641876 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 282/550] 10.222.0.36 () {56 vars in 1135 bytes} [Wed Jul 27 10:35:00 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:35:00.645571 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213e38f7-f5a8-450d-87b3-1aecd95d835f None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:00.670661 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 269/551] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 10:35:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 27 10:35:00.701114 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619a31cc-4645-4580-8714-819c56f3000f None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:00.723451 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:00.816875 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213e38f7-f5a8-450d-87b3-1aecd95d835f None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:00.846900 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:00.850517 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619a31cc-4645-4580-8714-819c56f3000f None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:00.861405 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d635aac-2701-4a7d-aaab-eeef7f4c3cf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:00.862309 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d635aac-2701-4a7d-aaab-eeef7f4c3cf6 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:00.862898 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d635aac-2701-4a7d-aaab-eeef7f4c3cf6 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:00.863697 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d635aac-2701-4a7d-aaab-eeef7f4c3cf6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:00.895241 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fc8936-6527-4b79-9c8c-c4e655e4688b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:00.895895 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fc8936-6527-4b79-9c8c-c4e655e4688b None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:00.896181 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fc8936-6527-4b79-9c8c-c4e655e4688b None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:00.896460 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fc8936-6527-4b79-9c8c-c4e655e4688b None admin] PATH_INFO: `/v3/roles/service` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:00.910639 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fc8936-6527-4b79-9c8c-c4e655e4688b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 10:35:00.916642 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application [None req-92fc8936-6527-4b79-9c8c-c4e655e4688b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 27 10:35:00.916642 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:35:00.916642 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:35:00.916642 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:35:00.916642 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:35:00.916642 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:35:00.916642 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:35:00.916642 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:35:00.916642 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:35:00.916642 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:35:00.916642 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:35:00.916642 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:35:00.916642 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 10:35:00.916642 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 10:35:00.916642 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 10:35:00.916642 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 10:35:00.917544 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 10:35:00.917544 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 10:35:00.917544 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 10:35:00.917544 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 10:35:00.917544 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:35:00.917544 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:35:00.917544 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:35:00.917544 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:35:00.917544 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:35:00.917544 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:35:00.917544 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:35:00.917544 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application with Lock( Jul 27 10:35:00.917544 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:35:00.917544 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:35:00.917544 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:35:00.917544 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:35:00.918441 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:35:00.918441 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:35:00.918441 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:35:00.918441 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:35:00.918441 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 10:35:00.918441 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 10:35:00.918441 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 10:35:00.918441 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 10:35:00.918441 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 10:35:00.918441 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 10:35:00.918441 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 27 10:35:00.918441 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Jul 27 10:35:00.918441 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 270/552] 10.222.0.36 () {56 vars in 1102 bytes} [Wed Jul 27 10:35:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 10:35:00.919683 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd7c95f-17c6-45a4-aea4-93f6417efd8a None None] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:00.919683 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd7c95f-17c6-45a4-aea4-93f6417efd8a None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:00.919920 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd7c95f-17c6-45a4-aea4-93f6417efd8a None None] PATH_INFO: `/` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:00.925925 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 271/553] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 10:35:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 10:35:00.925925 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd7c95f-17c6-45a4-aea4-93f6417efd8a None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:00.968289 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d635aac-2701-4a7d-aaab-eeef7f4c3cf6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:00.970365 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d635aac-2701-4a7d-aaab-eeef7f4c3cf6 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:01.019424 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:01.046117 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd7c95f-17c6-45a4-aea4-93f6417efd8a None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:01.083047 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dded960d-237b-4201-997d-af23385db613 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:01.083880 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dded960d-237b-4201-997d-af23385db613 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:01.084324 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dded960d-237b-4201-997d-af23385db613 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:01.084680 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dded960d-237b-4201-997d-af23385db613 None admin] PATH_INFO: `/v3/roles` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:01.085667 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dded960d-237b-4201-997d-af23385db613 None admin] RBAC: Adding query filter params (name=service) {{(pid=86828) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:35:01.086078 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dded960d-237b-4201-997d-af23385db613 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:01.087632 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dded960d-237b-4201-997d-af23385db613 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:01.096624 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 272/554] 10.222.0.36 () {56 vars in 1103 bytes} [Wed Jul 27 10:35:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:35:01.098080 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f09dff5-671b-43b7-8c41-10962940b971 None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:01.098525 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f09dff5-671b-43b7-8c41-10962940b971 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:01.099034 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f09dff5-671b-43b7-8c41-10962940b971 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:01.108448 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 283/555] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jul 27 10:35:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 463 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 10:35:01.109470 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d635aac-2701-4a7d-aaab-eeef7f4c3cf6 None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:01.119844 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-6f09dff5-671b-43b7-8c41-10962940b971 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:35:01.121199 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-6f09dff5-671b-43b7-8c41-10962940b971 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:35:01.140487 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:01.197910 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 273/556] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 10:35:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 10:35:01.198785 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f09dff5-671b-43b7-8c41-10962940b971 None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:01.231940 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d635aac-2701-4a7d-aaab-eeef7f4c3cf6 None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:01.267574 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9dc92b-d01d-4804-8604-d23904f0daaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:01.268304 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9dc92b-d01d-4804-8604-d23904f0daaa None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:01.268593 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9dc92b-d01d-4804-8604-d23904f0daaa None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:01.268868 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9dc92b-d01d-4804-8604-d23904f0daaa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:01.275264 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:01.280442 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9dc92b-d01d-4804-8604-d23904f0daaa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:01.281906 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9dc92b-d01d-4804-8604-d23904f0daaa None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:01.297571 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application [None req-6f9dc92b-d01d-4804-8604-d23904f0daaa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 10:35:01.297571 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:35:01.297571 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:35:01.297571 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:35:01.297571 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:35:01.297571 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:35:01.297571 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:35:01.297571 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:35:01.297571 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:35:01.297571 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:35:01.297571 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:35:01.297571 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:35:01.297571 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 10:35:01.297571 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 10:35:01.297571 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 10:35:01.298429 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 10:35:01.298429 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:35:01.298429 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:35:01.298429 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:35:01.298429 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:35:01.298429 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:35:01.298429 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:35:01.298429 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:35:01.298429 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application with Lock( Jul 27 10:35:01.298429 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:35:01.298429 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:35:01.298429 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:35:01.298429 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:35:01.298429 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:35:01.298429 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:35:01.298429 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:35:01.298429 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:35:01.299317 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 10:35:01.299317 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 10:35:01.299317 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 10:35:01.299317 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Jul 27 10:35:01.300804 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 284/557] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 10:35:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 10:35:01.302107 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e74bf0-133e-4f28-a57d-82f3633e844b None None] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:01.302539 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e74bf0-133e-4f28-a57d-82f3633e844b None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:01.302996 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e74bf0-133e-4f28-a57d-82f3633e844b None None] PATH_INFO: `/` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:01.304732 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 285/558] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 10:35:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 10:35:01.305570 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e74bf0-133e-4f28-a57d-82f3633e844b None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:01.316738 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f09dff5-671b-43b7-8c41-10962940b971 None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:01.353413 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32375e93-6393-4b92-b99d-81bcc470aa26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:01.354215 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32375e93-6393-4b92-b99d-81bcc470aa26 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:01.354501 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32375e93-6393-4b92-b99d-81bcc470aa26 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:01.354807 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32375e93-6393-4b92-b99d-81bcc470aa26 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:01.366685 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32375e93-6393-4b92-b99d-81bcc470aa26 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:01.368224 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32375e93-6393-4b92-b99d-81bcc470aa26 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:01.384439 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application [None req-32375e93-6393-4b92-b99d-81bcc470aa26 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 10:35:01.384439 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:35:01.384439 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:35:01.384439 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:35:01.384439 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:35:01.384439 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:35:01.384439 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:35:01.384439 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:35:01.384439 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:35:01.384439 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:35:01.384439 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:35:01.384439 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:35:01.384439 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 10:35:01.384439 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 10:35:01.384439 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 10:35:01.385782 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 10:35:01.385782 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:35:01.385782 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:35:01.385782 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:35:01.385782 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:35:01.385782 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:35:01.385782 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:35:01.385782 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:35:01.385782 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application with Lock( Jul 27 10:35:01.385782 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:35:01.385782 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:35:01.385782 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:35:01.385782 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:35:01.385782 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:35:01.385782 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:35:01.385782 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:35:01.385782 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:35:01.386692 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 10:35:01.386692 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 10:35:01.386692 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 10:35:01.386692 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Jul 27 10:35:01.388222 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:01.390480 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 274/559] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 10:35:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 10:35:01.391287 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32375e93-6393-4b92-b99d-81bcc470aa26 None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:01.416385 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e74bf0-133e-4f28-a57d-82f3633e844b None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:01.452758 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87418417-54de-4352-a038-5ff6bbe5fb61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:01.453519 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87418417-54de-4352-a038-5ff6bbe5fb61 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:01.453807 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87418417-54de-4352-a038-5ff6bbe5fb61 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:01.454088 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87418417-54de-4352-a038-5ff6bbe5fb61 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:01.464985 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87418417-54de-4352-a038-5ff6bbe5fb61 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 10:35:01.470385 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application [None req-87418417-54de-4352-a038-5ff6bbe5fb61 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 27 10:35:01.470385 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:35:01.470385 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:35:01.470385 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:35:01.470385 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:35:01.470385 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:35:01.470385 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:35:01.470385 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:35:01.470385 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:35:01.470385 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:35:01.470385 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:35:01.470385 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:35:01.470385 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 10:35:01.470385 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 10:35:01.470385 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 10:35:01.470385 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 10:35:01.471354 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 10:35:01.471354 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 10:35:01.471354 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 10:35:01.471354 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 10:35:01.471354 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:35:01.471354 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:35:01.471354 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:35:01.471354 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:35:01.471354 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:35:01.471354 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:35:01.471354 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:35:01.471354 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application with Lock( Jul 27 10:35:01.471354 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:35:01.471354 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:35:01.471354 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:35:01.471354 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:35:01.472164 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:35:01.472164 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:35:01.472164 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:35:01.472164 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:35:01.472164 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 10:35:01.472164 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 10:35:01.472164 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 10:35:01.472164 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 10:35:01.472164 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 10:35:01.472164 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 10:35:01.472164 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 27 10:35:01.472164 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Jul 27 10:35:01.474905 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 286/560] 10.222.0.36 () {56 vars in 1102 bytes} [Wed Jul 27 10:35:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 10:35:01.476951 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c492d12-1649-4b89-96fb-636caad003d2 None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:01.477662 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c492d12-1649-4b89-96fb-636caad003d2 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:01.477949 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c492d12-1649-4b89-96fb-636caad003d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:01.498376 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-8c492d12-1649-4b89-96fb-636caad003d2 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:35:01.499606 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-8c492d12-1649-4b89-96fb-636caad003d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:35:01.500685 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32375e93-6393-4b92-b99d-81bcc470aa26 None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:01.525220 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:01.538451 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31838e73-af9d-481d-8a7c-eb2d5a9b21f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:01.539052 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31838e73-af9d-481d-8a7c-eb2d5a9b21f5 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:01.539329 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31838e73-af9d-481d-8a7c-eb2d5a9b21f5 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:01.539600 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31838e73-af9d-481d-8a7c-eb2d5a9b21f5 None admin] PATH_INFO: `/v3/domains` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:01.540295 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31838e73-af9d-481d-8a7c-eb2d5a9b21f5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86828) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:35:01.540604 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31838e73-af9d-481d-8a7c-eb2d5a9b21f5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:01.541979 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31838e73-af9d-481d-8a7c-eb2d5a9b21f5 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:01.558643 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 275/561] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 10:35:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:35:01.559543 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31838e73-af9d-481d-8a7c-eb2d5a9b21f5 None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:01.582504 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 287/562] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 10:35:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 10:35:01.583370 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c492d12-1649-4b89-96fb-636caad003d2 None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:01.648141 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:01.683373 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31838e73-af9d-481d-8a7c-eb2d5a9b21f5 None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:01.687373 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c492d12-1649-4b89-96fb-636caad003d2 None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:01.722852 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ef7164-392e-4b7c-aae2-e2068bc4304b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:01.723513 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ef7164-392e-4b7c-aae2-e2068bc4304b None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:01.723798 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ef7164-392e-4b7c-aae2-e2068bc4304b None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:01.723897 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ad9833-0b79-430e-8c6d-bf19e5606817 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:01.724106 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ef7164-392e-4b7c-aae2-e2068bc4304b None admin] PATH_INFO: `/v3/domains` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:01.724464 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ad9833-0b79-430e-8c6d-bf19e5606817 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:01.724753 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ad9833-0b79-430e-8c6d-bf19e5606817 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:01.724847 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ef7164-392e-4b7c-aae2-e2068bc4304b None admin] RBAC: Adding query filter params (name=Default) {{(pid=86828) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:35:01.725032 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ad9833-0b79-430e-8c6d-bf19e5606817 None admin] PATH_INFO: `/v3/roles` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:01.725143 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ef7164-392e-4b7c-aae2-e2068bc4304b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:01.726282 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ad9833-0b79-430e-8c6d-bf19e5606817 None admin] RBAC: Adding query filter params (name=service) {{(pid=86827) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:35:01.726608 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ad9833-0b79-430e-8c6d-bf19e5606817 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:01.727001 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ef7164-392e-4b7c-aae2-e2068bc4304b None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:01.727972 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ad9833-0b79-430e-8c6d-bf19e5606817 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:01.740860 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 288/563] 10.222.0.36 () {56 vars in 1103 bytes} [Wed Jul 27 10:35:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:35:01.740860 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ad9833-0b79-430e-8c6d-bf19e5606817 None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:01.744574 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 276/564] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 10:35:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:35:01.748925 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ef7164-392e-4b7c-aae2-e2068bc4304b None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:01.794475 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:01.848264 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ad9833-0b79-430e-8c6d-bf19e5606817 None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:01.861279 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ef7164-392e-4b7c-aae2-e2068bc4304b None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:01.877143 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:01.887100 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdea0a4-1f66-491c-8bec-d0ae560d450c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:01.887752 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdea0a4-1f66-491c-8bec-d0ae560d450c None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:01.888039 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdea0a4-1f66-491c-8bec-d0ae560d450c None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:01.888370 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdea0a4-1f66-491c-8bec-d0ae560d450c None admin] PATH_INFO: `/v3/users/glance` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:01.896277 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acdea0a4-1f66-491c-8bec-d0ae560d450c None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:01.897744 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acdea0a4-1f66-491c-8bec-d0ae560d450c None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:01.899105 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a3d46f-7fd2-4e11-8e1f-8318fe4ea96a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:01.899754 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a3d46f-7fd2-4e11-8e1f-8318fe4ea96a None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:01.900038 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a3d46f-7fd2-4e11-8e1f-8318fe4ea96a None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:01.900601 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a3d46f-7fd2-4e11-8e1f-8318fe4ea96a None admin] PATH_INFO: `/v3/roles/service` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:01.909157 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application [None req-acdea0a4-1f66-491c-8bec-d0ae560d450c None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 27 10:35:01.909157 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:35:01.909157 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:35:01.909157 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:35:01.909157 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:35:01.909157 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:35:01.909157 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:35:01.909157 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:35:01.909157 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:35:01.909157 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:35:01.909157 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:35:01.909157 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:35:01.909157 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 10:35:01.909157 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 10:35:01.909157 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 10:35:01.909157 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 10:35:01.910131 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:35:01.910131 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:35:01.910131 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 10:35:01.910131 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 10:35:01.910131 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 10:35:01.910131 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 10:35:01.910131 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:35:01.910131 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:35:01.910131 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:35:01.910131 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:35:01.910131 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:35:01.910131 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application with Lock( Jul 27 10:35:01.910131 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:35:01.910131 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:35:01.910131 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:35:01.910131 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:35:01.910131 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:35:01.910131 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:35:01.911320 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:35:01.911320 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:35:01.911320 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 10:35:01.911320 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 10:35:01.911320 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 10:35:01.911320 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 10:35:01.911320 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 10:35:01.911320 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 10:35:01.911320 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 27 10:35:01.911320 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Jul 27 10:35:01.911320 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 289/565] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jul 27 10:35:01 2022] GET /identity/v3/users/glance => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 10:35:01.911951 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdea0a4-1f66-491c-8bec-d0ae560d450c None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:01.916975 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a3d46f-7fd2-4e11-8e1f-8318fe4ea96a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 10:35:01.922989 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application [None req-47a3d46f-7fd2-4e11-8e1f-8318fe4ea96a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 27 10:35:01.922989 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:35:01.922989 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:35:01.922989 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:35:01.922989 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:35:01.922989 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:35:01.922989 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:35:01.922989 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:35:01.922989 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:35:01.922989 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:35:01.922989 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:35:01.922989 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:35:01.922989 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 10:35:01.922989 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 10:35:01.922989 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 10:35:01.922989 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 10:35:01.923909 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 10:35:01.923909 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 10:35:01.923909 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 10:35:01.923909 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 10:35:01.923909 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:35:01.923909 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:35:01.923909 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:35:01.923909 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:35:01.923909 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:35:01.923909 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:35:01.923909 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:35:01.923909 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application with Lock( Jul 27 10:35:01.923909 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:35:01.923909 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:35:01.923909 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:35:01.923909 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:35:01.924752 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:35:01.924752 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:35:01.924752 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:35:01.924752 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:35:01.924752 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 10:35:01.924752 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 10:35:01.924752 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 10:35:01.924752 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 10:35:01.924752 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 10:35:01.924752 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 10:35:01.924752 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 27 10:35:01.924752 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Jul 27 10:35:01.926273 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 277/566] 10.222.0.36 () {56 vars in 1102 bytes} [Wed Jul 27 10:35:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 10:35:01.927382 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a3d46f-7fd2-4e11-8e1f-8318fe4ea96a None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:02.031885 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdea0a4-1f66-491c-8bec-d0ae560d450c None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:02.046293 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a3d46f-7fd2-4e11-8e1f-8318fe4ea96a None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:02.048691 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:02.068121 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593e437e-b703-4d45-9961-24d045222452 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:02.068760 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593e437e-b703-4d45-9961-24d045222452 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:02.069041 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593e437e-b703-4d45-9961-24d045222452 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:02.069346 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593e437e-b703-4d45-9961-24d045222452 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:02.080885 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593e437e-b703-4d45-9961-24d045222452 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:02.082333 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593e437e-b703-4d45-9961-24d045222452 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:02.089917 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e57b2ce-c0c2-4203-a030-80952e8fe5ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:02.090561 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e57b2ce-c0c2-4203-a030-80952e8fe5ae None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:02.090880 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e57b2ce-c0c2-4203-a030-80952e8fe5ae None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:02.091156 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e57b2ce-c0c2-4203-a030-80952e8fe5ae None admin] PATH_INFO: `/v3/projects/service` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:02.097372 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application [None req-593e437e-b703-4d45-9961-24d045222452 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 10:35:02.097372 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:35:02.097372 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:35:02.097372 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:35:02.097372 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:35:02.097372 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:35:02.097372 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:35:02.097372 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:35:02.097372 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:35:02.097372 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:35:02.097372 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:35:02.097372 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:35:02.097372 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 10:35:02.097372 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 10:35:02.097372 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 10:35:02.098288 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 10:35:02.098288 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:35:02.098288 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:35:02.098288 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:35:02.098288 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:35:02.098288 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:35:02.098288 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:35:02.098288 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:35:02.098288 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application with Lock( Jul 27 10:35:02.098288 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:35:02.098288 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:35:02.098288 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:35:02.098288 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:35:02.098288 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:35:02.098288 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:35:02.098288 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:35:02.098288 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:35:02.099206 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 10:35:02.099206 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 10:35:02.099206 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 10:35:02.099206 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Jul 27 10:35:02.099419 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e57b2ce-c0c2-4203-a030-80952e8fe5ae None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:02.099487 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 290/567] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 10:35:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 10:35:02.099561 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e57b2ce-c0c2-4203-a030-80952e8fe5ae None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:02.100145 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593e437e-b703-4d45-9961-24d045222452 None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:02.112725 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application [None req-1e57b2ce-c0c2-4203-a030-80952e8fe5ae None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 10:35:02.112725 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:35:02.112725 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:35:02.112725 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:35:02.112725 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:35:02.112725 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:35:02.112725 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:35:02.112725 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:35:02.112725 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:35:02.112725 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:35:02.112725 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:35:02.112725 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:35:02.112725 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 10:35:02.112725 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 10:35:02.112725 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 10:35:02.113600 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 10:35:02.113600 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:35:02.113600 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:35:02.113600 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:35:02.113600 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:35:02.113600 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:35:02.113600 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:35:02.113600 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:35:02.113600 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application with Lock( Jul 27 10:35:02.113600 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:35:02.113600 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:35:02.113600 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:35:02.113600 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:35:02.113600 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:35:02.113600 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:35:02.113600 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:35:02.113600 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:35:02.114458 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 10:35:02.114458 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 10:35:02.114458 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 10:35:02.114458 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 10:35:02.114458 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 10:35:02.114458 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 10:35:02.114458 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 10:35:02.114458 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Jul 27 10:35:02.114458 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 278/568] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Jul 27 10:35:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 10:35:02.114978 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e57b2ce-c0c2-4203-a030-80952e8fe5ae None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:02.144985 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:02.204149 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593e437e-b703-4d45-9961-24d045222452 None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:02.215311 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e57b2ce-c0c2-4203-a030-80952e8fe5ae None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:02.237329 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c77a0e5-6fa4-49e4-a2f4-aea3b265aa03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:02.237946 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c77a0e5-6fa4-49e4-a2f4-aea3b265aa03 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:02.238225 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c77a0e5-6fa4-49e4-a2f4-aea3b265aa03 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:02.238496 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c77a0e5-6fa4-49e4-a2f4-aea3b265aa03 None admin] PATH_INFO: `/v3/users` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:02.239314 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c77a0e5-6fa4-49e4-a2f4-aea3b265aa03 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=86827) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:35:02.239620 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c77a0e5-6fa4-49e4-a2f4-aea3b265aa03 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:02.241029 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c77a0e5-6fa4-49e4-a2f4-aea3b265aa03 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:02.247935 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56a3da8-87b1-4915-9500-99c52c668932 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:02.248548 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56a3da8-87b1-4915-9500-99c52c668932 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:02.248835 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56a3da8-87b1-4915-9500-99c52c668932 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:02.249113 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56a3da8-87b1-4915-9500-99c52c668932 None admin] PATH_INFO: `/v3/roles` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:02.249821 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56a3da8-87b1-4915-9500-99c52c668932 None admin] RBAC: Adding query filter params (name=service) {{(pid=86828) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:35:02.250127 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56a3da8-87b1-4915-9500-99c52c668932 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:02.251542 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56a3da8-87b1-4915-9500-99c52c668932 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:02.257799 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 291/569] 10.222.0.36 () {56 vars in 1137 bytes} [Wed Jul 27 10:35:02 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:35:02.259248 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1972a289-5142-4aaf-8964-5925e99af659 None None] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:02.263130 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1972a289-5142-4aaf-8964-5925e99af659 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:02.263258 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 279/570] 10.222.0.36 () {56 vars in 1103 bytes} [Wed Jul 27 10:35:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:35:02.263338 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1972a289-5142-4aaf-8964-5925e99af659 None None] PATH_INFO: `/` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:02.263437 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56a3da8-87b1-4915-9500-99c52c668932 None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:02.263512 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 292/571] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 10:35:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 10:35:02.263512 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1972a289-5142-4aaf-8964-5925e99af659 None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:02.291920 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:02.356822 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:02.363872 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1972a289-5142-4aaf-8964-5925e99af659 None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:02.364407 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56a3da8-87b1-4915-9500-99c52c668932 None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:02.396709 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fdc251-c305-4bbc-b37e-ac468b3bd161 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:02.397041 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e468b31e-b291-4e61-a68f-bfcc4e48ee95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:02.397806 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fdc251-c305-4bbc-b37e-ac468b3bd161 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:02.398089 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fdc251-c305-4bbc-b37e-ac468b3bd161 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:02.398183 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e468b31e-b291-4e61-a68f-bfcc4e48ee95 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:02.398360 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fdc251-c305-4bbc-b37e-ac468b3bd161 None admin] PATH_INFO: `/v3/projects` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:02.398450 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e468b31e-b291-4e61-a68f-bfcc4e48ee95 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:02.398711 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e468b31e-b291-4e61-a68f-bfcc4e48ee95 None admin] PATH_INFO: `/v3/domains` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:02.399147 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fdc251-c305-4bbc-b37e-ac468b3bd161 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86827) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:35:02.399394 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e468b31e-b291-4e61-a68f-bfcc4e48ee95 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86828) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:35:02.399483 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fdc251-c305-4bbc-b37e-ac468b3bd161 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:02.399695 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e468b31e-b291-4e61-a68f-bfcc4e48ee95 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:02.400856 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fdc251-c305-4bbc-b37e-ac468b3bd161 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:02.401028 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e468b31e-b291-4e61-a68f-bfcc4e48ee95 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:02.412008 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 293/572] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Jul 27 10:35:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:35:02.417855 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 280/573] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 10:35:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:35:02.417987 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fdc251-c305-4bbc-b37e-ac468b3bd161 None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:02.418065 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e468b31e-b291-4e61-a68f-bfcc4e48ee95 None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:02.496132 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:02.512129 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e468b31e-b291-4e61-a68f-bfcc4e48ee95 None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:02.515497 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fdc251-c305-4bbc-b37e-ac468b3bd161 None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:02.545427 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0180c656-5bb0-4116-a368-a55fe745115d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:02.546054 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0180c656-5bb0-4116-a368-a55fe745115d None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:02.546332 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0180c656-5bb0-4116-a368-a55fe745115d None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:02.546623 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0180c656-5bb0-4116-a368-a55fe745115d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:02.548590 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419b4365-d701-4d45-8243-0a99ec7f8820 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:02.549222 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419b4365-d701-4d45-8243-0a99ec7f8820 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:02.549505 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419b4365-d701-4d45-8243-0a99ec7f8820 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:02.549783 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419b4365-d701-4d45-8243-0a99ec7f8820 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:02.556255 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0180c656-5bb0-4116-a368-a55fe745115d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:02.557685 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0180c656-5bb0-4116-a368-a55fe745115d None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:02.560121 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419b4365-d701-4d45-8243-0a99ec7f8820 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:02.561549 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419b4365-d701-4d45-8243-0a99ec7f8820 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:02.573675 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application [None req-0180c656-5bb0-4116-a368-a55fe745115d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 10:35:02.573675 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:35:02.573675 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:35:02.573675 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:35:02.573675 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:35:02.573675 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:35:02.573675 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:35:02.573675 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:35:02.573675 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:35:02.573675 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:35:02.573675 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:35:02.573675 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:35:02.573675 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 10:35:02.573675 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 10:35:02.573675 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 10:35:02.574761 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 10:35:02.574761 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:35:02.574761 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:35:02.574761 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:35:02.574761 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:35:02.574761 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:35:02.574761 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:35:02.574761 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:35:02.574761 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application with Lock( Jul 27 10:35:02.574761 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:35:02.574761 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:35:02.574761 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:35:02.574761 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:35:02.574761 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:35:02.574761 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:35:02.574761 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:35:02.574761 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:35:02.575698 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 10:35:02.575698 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 10:35:02.575698 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 10:35:02.575698 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Jul 27 10:35:02.575698 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 281/574] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 10:35:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 10:35:02.576432 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c782781-f293-4ea4-af51-213772d56d3e None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:02.576714 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c782781-f293-4ea4-af51-213772d56d3e None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:02.576988 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c782781-f293-4ea4-af51-213772d56d3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:02.579607 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application [None req-419b4365-d701-4d45-8243-0a99ec7f8820 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 10:35:02.579607 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:35:02.579607 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:35:02.579607 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:35:02.579607 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:35:02.579607 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:35:02.579607 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:35:02.579607 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:35:02.579607 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:35:02.579607 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:35:02.579607 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:35:02.579607 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:35:02.579607 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 10:35:02.579607 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 10:35:02.579607 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 10:35:02.580441 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 10:35:02.580441 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:35:02.580441 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:35:02.580441 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:35:02.580441 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:35:02.580441 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:35:02.580441 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:35:02.580441 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:35:02.580441 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application with Lock( Jul 27 10:35:02.580441 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:35:02.580441 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:35:02.580441 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:35:02.580441 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:35:02.580441 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:35:02.580441 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:35:02.580441 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:35:02.580441 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:35:02.581342 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 10:35:02.581342 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 10:35:02.581342 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 10:35:02.581342 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Jul 27 10:35:02.581342 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 294/575] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 10:35:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 10:35:02.581614 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419b4365-d701-4d45-8243-0a99ec7f8820 None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:02.596463 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:02.600625 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-6c782781-f293-4ea4-af51-213772d56d3e None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:35:02.601826 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-6c782781-f293-4ea4-af51-213772d56d3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:35:02.667774 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 282/576] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 10:35:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 10:35:02.668524 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c782781-f293-4ea4-af51-213772d56d3e None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:02.685841 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419b4365-d701-4d45-8243-0a99ec7f8820 None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:02.719928 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fceb02c-d7a3-4ebf-b662-48bf8674c3aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:02.720573 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fceb02c-d7a3-4ebf-b662-48bf8674c3aa None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:02.720859 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fceb02c-d7a3-4ebf-b662-48bf8674c3aa None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:02.721135 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fceb02c-d7a3-4ebf-b662-48bf8674c3aa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:02.721919 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fceb02c-d7a3-4ebf-b662-48bf8674c3aa None admin] RBAC: Adding query filter params (role.id=9e0038c6bbd34556af3f84f9e2a7c2c5, scope.project.id=5f296d8e3b8f4001909addc49d8dac01, user.id=89ac506eefcd4e42a84b44fa15fb3975) {{(pid=86827) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:35:02.722229 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fceb02c-d7a3-4ebf-b662-48bf8674c3aa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:02.723703 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fceb02c-d7a3-4ebf-b662-48bf8674c3aa None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:02.729053 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 295/577] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Jul 27 10:35:02 2022] GET /identity/v3/role_assignments?user.id=89ac506eefcd4e42a84b44fa15fb3975&scope.project.id=5f296d8e3b8f4001909addc49d8dac01&role.id=9e0038c6bbd34556af3f84f9e2a7c2c5 => generated 259 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:35:02.729821 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fceb02c-d7a3-4ebf-b662-48bf8674c3aa None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:02.751469 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:02.796332 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c782781-f293-4ea4-af51-213772d56d3e None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:02.825489 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:02.836998 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cebe5c-5732-45df-b6dc-8fa5c815b4f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:02.837654 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cebe5c-5732-45df-b6dc-8fa5c815b4f1 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:02.837935 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cebe5c-5732-45df-b6dc-8fa5c815b4f1 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:02.838210 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cebe5c-5732-45df-b6dc-8fa5c815b4f1 None admin] PATH_INFO: `/v3/users/nova` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:02.846828 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cebe5c-5732-45df-b6dc-8fa5c815b4f1 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:02.848303 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cebe5c-5732-45df-b6dc-8fa5c815b4f1 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:02.861408 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application [None req-e2cebe5c-5732-45df-b6dc-8fa5c815b4f1 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 27 10:35:02.861408 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:35:02.861408 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:35:02.861408 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:35:02.861408 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:35:02.861408 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:35:02.861408 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:35:02.861408 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:35:02.861408 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:35:02.861408 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:35:02.861408 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:35:02.861408 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:35:02.861408 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 10:35:02.861408 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 10:35:02.861408 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 10:35:02.861408 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 10:35:02.868197 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:35:02.868197 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:35:02.868197 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 10:35:02.868197 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 10:35:02.868197 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 10:35:02.868197 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 10:35:02.868197 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:35:02.868197 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:35:02.868197 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:35:02.868197 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:35:02.868197 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:35:02.868197 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application with Lock( Jul 27 10:35:02.868197 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:35:02.868197 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:35:02.868197 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:35:02.868197 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:35:02.868197 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:35:02.868197 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:35:02.869415 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:35:02.869415 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:35:02.869415 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 10:35:02.869415 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 10:35:02.869415 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 10:35:02.869415 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 10:35:02.869415 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 10:35:02.869415 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 10:35:02.869415 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 27 10:35:02.869415 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Jul 27 10:35:02.870072 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fceb02c-d7a3-4ebf-b662-48bf8674c3aa None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:02.870159 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 283/578] 10.222.0.36 () {56 vars in 1093 bytes} [Wed Jul 27 10:35:02 2022] GET /identity/v3/users/nova => generated 82 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 10:35:02.870159 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cebe5c-5732-45df-b6dc-8fa5c815b4f1 None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:02.907763 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bbfcc8-582a-4be3-9800-c329e65a39c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:02.908664 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bbfcc8-582a-4be3-9800-c329e65a39c5 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:02.909104 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bbfcc8-582a-4be3-9800-c329e65a39c5 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:02.909383 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bbfcc8-582a-4be3-9800-c329e65a39c5 None admin] PATH_INFO: `/v3/domains` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:02.910085 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bbfcc8-582a-4be3-9800-c329e65a39c5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86827) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:35:02.910393 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bbfcc8-582a-4be3-9800-c329e65a39c5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:02.911865 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bbfcc8-582a-4be3-9800-c329e65a39c5 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:02.923975 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 296/579] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 10:35:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:35:02.929506 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bbfcc8-582a-4be3-9800-c329e65a39c5 None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:02.986295 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cebe5c-5732-45df-b6dc-8fa5c815b4f1 None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:03.013074 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:03.023659 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048ac61d-7d27-459f-ac08-d8bda1b7ad12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:03.024295 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048ac61d-7d27-459f-ac08-d8bda1b7ad12 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:03.024580 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048ac61d-7d27-459f-ac08-d8bda1b7ad12 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:03.024873 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048ac61d-7d27-459f-ac08-d8bda1b7ad12 None admin] PATH_INFO: `/v3/domains` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:03.025560 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048ac61d-7d27-459f-ac08-d8bda1b7ad12 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86828) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:35:03.025966 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048ac61d-7d27-459f-ac08-d8bda1b7ad12 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:03.027294 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048ac61d-7d27-459f-ac08-d8bda1b7ad12 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:03.044374 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 284/580] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 10:35:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:35:03.044374 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048ac61d-7d27-459f-ac08-d8bda1b7ad12 None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:03.044645 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bbfcc8-582a-4be3-9800-c329e65a39c5 None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:03.076937 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:03.082645 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243219f3-d552-4004-980e-e7bed4aa5c46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:03.083299 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243219f3-d552-4004-980e-e7bed4aa5c46 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:03.083582 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243219f3-d552-4004-980e-e7bed4aa5c46 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:03.083866 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243219f3-d552-4004-980e-e7bed4aa5c46 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:03.094186 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243219f3-d552-4004-980e-e7bed4aa5c46 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 10:35:03.099235 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application [None req-243219f3-d552-4004-980e-e7bed4aa5c46 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 27 10:35:03.099235 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:35:03.099235 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:35:03.099235 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:35:03.099235 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:35:03.099235 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:35:03.099235 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:35:03.099235 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:35:03.099235 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:35:03.099235 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:35:03.099235 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:35:03.099235 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:35:03.099235 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 10:35:03.099235 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 10:35:03.099235 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 10:35:03.099235 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 10:35:03.100180 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 10:35:03.100180 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 10:35:03.100180 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 10:35:03.100180 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 10:35:03.100180 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:35:03.100180 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:35:03.100180 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:35:03.100180 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:35:03.100180 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:35:03.100180 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:35:03.100180 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:35:03.100180 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application with Lock( Jul 27 10:35:03.100180 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:35:03.100180 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:35:03.100180 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:35:03.100180 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:35:03.101046 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:35:03.101046 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:35:03.101046 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:35:03.101046 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:35:03.101046 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 10:35:03.101046 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 10:35:03.101046 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 10:35:03.101046 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 10:35:03.101046 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 10:35:03.101046 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 10:35:03.101046 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 27 10:35:03.101046 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Jul 27 10:35:03.101046 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 297/581] 10.222.0.36 () {56 vars in 1102 bytes} [Wed Jul 27 10:35:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 10:35:03.101809 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243219f3-d552-4004-980e-e7bed4aa5c46 None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:03.162001 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048ac61d-7d27-459f-ac08-d8bda1b7ad12 None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:03.198815 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab3109a-cb0b-40d5-b47a-242f66b57104 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:03.199370 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab3109a-cb0b-40d5-b47a-242f66b57104 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:03.199656 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab3109a-cb0b-40d5-b47a-242f66b57104 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:03.199937 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab3109a-cb0b-40d5-b47a-242f66b57104 None admin] PATH_INFO: `/v3/users` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:03.200745 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab3109a-cb0b-40d5-b47a-242f66b57104 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86828) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:35:03.201059 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab3109a-cb0b-40d5-b47a-242f66b57104 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:03.202487 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab3109a-cb0b-40d5-b47a-242f66b57104 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:03.212264 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243219f3-d552-4004-980e-e7bed4aa5c46 None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:03.224389 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 285/582] 10.222.0.36 () {56 vars in 1133 bytes} [Wed Jul 27 10:35:03 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:35:03.224389 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab3109a-cb0b-40d5-b47a-242f66b57104 None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:03.254022 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a082317a-0112-4ce1-beaf-99950ab70503 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:03.254951 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a082317a-0112-4ce1-beaf-99950ab70503 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:03.255406 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a082317a-0112-4ce1-beaf-99950ab70503 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:03.255683 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a082317a-0112-4ce1-beaf-99950ab70503 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:03.263479 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a082317a-0112-4ce1-beaf-99950ab70503 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:03.265030 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a082317a-0112-4ce1-beaf-99950ab70503 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:03.270777 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:03.278171 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application [None req-a082317a-0112-4ce1-beaf-99950ab70503 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 27 10:35:03.278171 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:35:03.278171 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:35:03.278171 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:35:03.278171 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:35:03.278171 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:35:03.278171 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:35:03.278171 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:35:03.278171 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:35:03.278171 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:35:03.278171 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:35:03.278171 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:35:03.278171 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 10:35:03.278171 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 10:35:03.278171 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 10:35:03.278171 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 10:35:03.279178 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:35:03.279178 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:35:03.279178 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 10:35:03.279178 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 10:35:03.279178 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 10:35:03.279178 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 10:35:03.279178 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:35:03.279178 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:35:03.279178 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:35:03.279178 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:35:03.279178 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:35:03.279178 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application with Lock( Jul 27 10:35:03.279178 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:35:03.279178 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:35:03.279178 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:35:03.279178 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:35:03.279178 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:35:03.279178 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:35:03.280093 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:35:03.280093 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:35:03.280093 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 10:35:03.280093 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 10:35:03.280093 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 10:35:03.280093 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 10:35:03.280093 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 10:35:03.280093 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 10:35:03.280093 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 27 10:35:03.280093 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Jul 27 10:35:03.280093 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 298/583] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jul 27 10:35:03 2022] GET /identity/v3/users/cinder => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 10:35:03.280093 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a082317a-0112-4ce1-beaf-99950ab70503 None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:03.330944 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab3109a-cb0b-40d5-b47a-242f66b57104 None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:03.337562 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:03.366969 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcbc82e-9d61-4581-91fc-647556d68035 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:03.367619 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcbc82e-9d61-4581-91fc-647556d68035 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:03.367907 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcbc82e-9d61-4581-91fc-647556d68035 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:03.368185 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcbc82e-9d61-4581-91fc-647556d68035 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:03.375067 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dcbc82e-9d61-4581-91fc-647556d68035 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:03.376561 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dcbc82e-9d61-4581-91fc-647556d68035 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:03.389007 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application [None req-6dcbc82e-9d61-4581-91fc-647556d68035 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 10:35:03.389007 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:35:03.389007 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:35:03.389007 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:35:03.389007 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:35:03.389007 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:35:03.389007 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:35:03.389007 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:35:03.389007 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:35:03.389007 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:35:03.389007 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:35:03.389007 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:35:03.389007 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 10:35:03.389007 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 10:35:03.389007 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 10:35:03.389921 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 10:35:03.389921 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:35:03.389921 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:35:03.389921 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:35:03.389921 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:35:03.389921 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:35:03.389921 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:35:03.389921 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:35:03.389921 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application with Lock( Jul 27 10:35:03.389921 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:35:03.389921 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:35:03.389921 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:35:03.389921 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:35:03.389921 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:35:03.389921 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:35:03.389921 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:35:03.389921 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:35:03.395873 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 10:35:03.395873 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 10:35:03.395873 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 10:35:03.395873 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 10:35:03.395873 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 10:35:03.395873 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 10:35:03.395873 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 10:35:03.395873 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Jul 27 10:35:03.396423 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a082317a-0112-4ce1-beaf-99950ab70503 None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:03.396511 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 286/584] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Jul 27 10:35:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 10:35:03.396511 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcbc82e-9d61-4581-91fc-647556d68035 None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:03.434442 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087ec59c-7b4c-419f-8a64-658d86e5e3cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:03.435262 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087ec59c-7b4c-419f-8a64-658d86e5e3cb None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:03.435664 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087ec59c-7b4c-419f-8a64-658d86e5e3cb None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:03.435969 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087ec59c-7b4c-419f-8a64-658d86e5e3cb None admin] PATH_INFO: `/v3/roles` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:03.436691 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087ec59c-7b4c-419f-8a64-658d86e5e3cb None admin] RBAC: Adding query filter params (name=service) {{(pid=86827) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:35:03.437003 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087ec59c-7b4c-419f-8a64-658d86e5e3cb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:03.438378 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087ec59c-7b4c-419f-8a64-658d86e5e3cb None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:03.451558 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 299/585] 10.222.0.36 () {56 vars in 1103 bytes} [Wed Jul 27 10:35:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:35:03.451558 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087ec59c-7b4c-419f-8a64-658d86e5e3cb None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:03.505055 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcbc82e-9d61-4581-91fc-647556d68035 None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:03.507648 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:03.544589 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c882cb8-2754-4914-b136-83b5c430ed84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:03.545228 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c882cb8-2754-4914-b136-83b5c430ed84 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:03.545512 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c882cb8-2754-4914-b136-83b5c430ed84 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:03.545786 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c882cb8-2754-4914-b136-83b5c430ed84 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:03.556249 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c882cb8-2754-4914-b136-83b5c430ed84 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:03.557742 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c882cb8-2754-4914-b136-83b5c430ed84 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:03.558673 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087ec59c-7b4c-419f-8a64-658d86e5e3cb None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:03.573015 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application [None req-4c882cb8-2754-4914-b136-83b5c430ed84 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 10:35:03.573015 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:35:03.573015 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:35:03.573015 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:35:03.573015 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:35:03.573015 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:35:03.573015 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:35:03.573015 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:35:03.573015 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:35:03.573015 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:35:03.573015 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:35:03.573015 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:35:03.573015 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 10:35:03.573015 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 10:35:03.573015 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 10:35:03.573919 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 10:35:03.573919 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:35:03.573919 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:35:03.573919 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:35:03.573919 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:35:03.573919 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:35:03.573919 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:35:03.573919 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:35:03.573919 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application with Lock( Jul 27 10:35:03.573919 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:35:03.573919 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:35:03.573919 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:35:03.573919 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:35:03.573919 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:35:03.573919 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:35:03.573919 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:35:03.573919 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:35:03.574853 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 10:35:03.574853 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 10:35:03.574853 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 10:35:03.574853 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Jul 27 10:35:03.574853 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 287/586] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 10:35:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 10:35:03.575153 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c882cb8-2754-4914-b136-83b5c430ed84 None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:03.576276 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:03.608360 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b865c81-9b94-4da4-9fc1-b541c0dfd534 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:03.608996 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b865c81-9b94-4da4-9fc1-b541c0dfd534 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:03.609475 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b865c81-9b94-4da4-9fc1-b541c0dfd534 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:03.609788 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b865c81-9b94-4da4-9fc1-b541c0dfd534 None admin] PATH_INFO: `/v3/users` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:03.610568 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b865c81-9b94-4da4-9fc1-b541c0dfd534 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=86827) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:35:03.610907 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b865c81-9b94-4da4-9fc1-b541c0dfd534 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:03.613555 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b865c81-9b94-4da4-9fc1-b541c0dfd534 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:03.632196 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 300/587] 10.222.0.36 () {56 vars in 1137 bytes} [Wed Jul 27 10:35:03 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:35:03.633013 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b865c81-9b94-4da4-9fc1-b541c0dfd534 None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:03.698978 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c882cb8-2754-4914-b136-83b5c430ed84 None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:03.735400 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb30ada-4d2d-4db6-a1f5-6e298fda0ed1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:03.735954 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb30ada-4d2d-4db6-a1f5-6e298fda0ed1 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:03.736240 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb30ada-4d2d-4db6-a1f5-6e298fda0ed1 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:03.736516 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb30ada-4d2d-4db6-a1f5-6e298fda0ed1 None admin] PATH_INFO: `/v3/projects` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:03.737274 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb30ada-4d2d-4db6-a1f5-6e298fda0ed1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86828) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:35:03.737585 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb30ada-4d2d-4db6-a1f5-6e298fda0ed1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:03.739062 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb30ada-4d2d-4db6-a1f5-6e298fda0ed1 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:03.743425 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b865c81-9b94-4da4-9fc1-b541c0dfd534 None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:03.756368 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 288/588] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Jul 27 10:35:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:35:03.756368 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb30ada-4d2d-4db6-a1f5-6e298fda0ed1 None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:03.770582 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:03.787839 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ab5c17-a6c7-4247-a536-2d3957d8ef82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:03.789028 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ab5c17-a6c7-4247-a536-2d3957d8ef82 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:03.789313 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ab5c17-a6c7-4247-a536-2d3957d8ef82 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:03.789581 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ab5c17-a6c7-4247-a536-2d3957d8ef82 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:03.800960 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ab5c17-a6c7-4247-a536-2d3957d8ef82 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:03.802448 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ab5c17-a6c7-4247-a536-2d3957d8ef82 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:03.818908 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application [None req-46ab5c17-a6c7-4247-a536-2d3957d8ef82 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 10:35:03.818908 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:35:03.818908 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:35:03.818908 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:35:03.818908 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:35:03.818908 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:35:03.818908 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:35:03.818908 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:35:03.818908 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:35:03.818908 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:35:03.818908 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:35:03.818908 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:35:03.818908 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 10:35:03.818908 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 10:35:03.818908 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 10:35:03.819905 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 10:35:03.819905 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:35:03.819905 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:35:03.819905 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:35:03.819905 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:35:03.819905 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:35:03.819905 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:35:03.819905 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:35:03.819905 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application with Lock( Jul 27 10:35:03.819905 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:35:03.819905 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:35:03.819905 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:35:03.819905 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:35:03.819905 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:35:03.819905 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:35:03.819905 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:35:03.819905 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:35:03.820897 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 10:35:03.820897 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 10:35:03.820897 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 10:35:03.820897 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Jul 27 10:35:03.820897 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 301/589] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 10:35:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 10:35:03.820897 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ab5c17-a6c7-4247-a536-2d3957d8ef82 None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:03.854639 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:03.879934 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb30ada-4d2d-4db6-a1f5-6e298fda0ed1 None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:03.928307 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e5c3ca-bc3b-424c-86ec-39290ff86110 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:03.929775 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e5c3ca-bc3b-424c-86ec-39290ff86110 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:03.929775 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e5c3ca-bc3b-424c-86ec-39290ff86110 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:03.930028 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e5c3ca-bc3b-424c-86ec-39290ff86110 None admin] PATH_INFO: `/v3/domains` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:03.930740 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e5c3ca-bc3b-424c-86ec-39290ff86110 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86828) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:35:03.931049 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e5c3ca-bc3b-424c-86ec-39290ff86110 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:03.932428 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e5c3ca-bc3b-424c-86ec-39290ff86110 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:03.938878 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ab5c17-a6c7-4247-a536-2d3957d8ef82 None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:03.958762 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 289/590] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 10:35:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:35:03.960040 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e5c3ca-bc3b-424c-86ec-39290ff86110 None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:04.001685 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c2e3a0-43ba-46dc-aae8-49be9d2848fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:04.002256 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c2e3a0-43ba-46dc-aae8-49be9d2848fc None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:04.002528 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c2e3a0-43ba-46dc-aae8-49be9d2848fc None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:04.002827 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c2e3a0-43ba-46dc-aae8-49be9d2848fc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:04.020782 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c2e3a0-43ba-46dc-aae8-49be9d2848fc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:04.023082 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c2e3a0-43ba-46dc-aae8-49be9d2848fc None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:04.044325 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application [None req-e2c2e3a0-43ba-46dc-aae8-49be9d2848fc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 10:35:04.044325 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:35:04.044325 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:35:04.044325 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:35:04.044325 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:35:04.044325 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:35:04.044325 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:35:04.044325 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:35:04.044325 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:35:04.044325 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:35:04.044325 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:35:04.044325 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:35:04.044325 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 10:35:04.044325 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 10:35:04.044325 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 10:35:04.045940 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 10:35:04.045940 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:35:04.045940 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:35:04.045940 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:35:04.045940 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:35:04.045940 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:35:04.045940 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:35:04.045940 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:35:04.045940 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application with Lock( Jul 27 10:35:04.045940 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:35:04.045940 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:35:04.045940 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:35:04.045940 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:35:04.045940 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:35:04.045940 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:35:04.045940 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:35:04.045940 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:35:04.047224 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 10:35:04.047224 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 10:35:04.047224 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 10:35:04.047224 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Jul 27 10:35:04.049199 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 302/591] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 10:35:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 10:35:04.050185 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c2e3a0-43ba-46dc-aae8-49be9d2848fc None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:04.067326 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:04.096674 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e5c3ca-bc3b-424c-86ec-39290ff86110 None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:04.147221 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:04.152767 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defca5f5-b304-456f-ac5f-d886703b62f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:04.153991 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defca5f5-b304-456f-ac5f-d886703b62f8 None admin] REQUEST_METHOD: `PUT` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:04.153991 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defca5f5-b304-456f-ac5f-d886703b62f8 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:04.153991 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defca5f5-b304-456f-ac5f-d886703b62f8 None admin] PATH_INFO: `/v3/projects/5f296d8e3b8f4001909addc49d8dac01/users/bc03c5d232654878aa1ddce15f49fdaa/roles/9e0038c6bbd34556af3f84f9e2a7c2c5` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:04.172543 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defca5f5-b304-456f-ac5f-d886703b62f8 None admin] RBAC: Authorizing `identity:create_grant(project_id=5f296d8e3b8f4001909addc49d8dac01, user_id=bc03c5d232654878aa1ddce15f49fdaa, role_id=9e0038c6bbd34556af3f84f9e2a7c2c5)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:04.173896 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defca5f5-b304-456f-ac5f-d886703b62f8 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:04.175324 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c2e3a0-43ba-46dc-aae8-49be9d2848fc None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:04.195909 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 290/592] 10.222.0.36 () {58 vars in 1439 bytes} [Wed Jul 27 10:35:03 2022] PUT /identity/v3/projects/5f296d8e3b8f4001909addc49d8dac01/users/bc03c5d232654878aa1ddce15f49fdaa/roles/9e0038c6bbd34556af3f84f9e2a7c2c5 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:35:04.197080 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defca5f5-b304-456f-ac5f-d886703b62f8 None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:04.221734 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd35bbec-c16e-4305-b06e-cec5de05c489 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:04.222398 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd35bbec-c16e-4305-b06e-cec5de05c489 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:04.222733 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd35bbec-c16e-4305-b06e-cec5de05c489 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:04.223015 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd35bbec-c16e-4305-b06e-cec5de05c489 None admin] PATH_INFO: `/v3/domains` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:04.223747 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd35bbec-c16e-4305-b06e-cec5de05c489 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86827) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:35:04.224132 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd35bbec-c16e-4305-b06e-cec5de05c489 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:04.226109 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd35bbec-c16e-4305-b06e-cec5de05c489 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:04.256753 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 303/593] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 10:35:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:35:04.256753 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd35bbec-c16e-4305-b06e-cec5de05c489 None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:04.359344 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defca5f5-b304-456f-ac5f-d886703b62f8 None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:04.391494 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:04.394248 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd35bbec-c16e-4305-b06e-cec5de05c489 None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:04.404119 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c462d8b-2f40-4b52-a595-dd27e6caf153 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:04.404766 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c462d8b-2f40-4b52-a595-dd27e6caf153 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:04.405076 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c462d8b-2f40-4b52-a595-dd27e6caf153 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:04.405349 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c462d8b-2f40-4b52-a595-dd27e6caf153 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:04.412932 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c462d8b-2f40-4b52-a595-dd27e6caf153 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:04.417217 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c462d8b-2f40-4b52-a595-dd27e6caf153 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:04.431079 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application [None req-1c462d8b-2f40-4b52-a595-dd27e6caf153 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 10:35:04.431079 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:35:04.431079 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:35:04.431079 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:35:04.431079 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:35:04.431079 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:35:04.431079 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:35:04.431079 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:35:04.431079 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:35:04.431079 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:35:04.431079 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:35:04.431079 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:35:04.431079 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 10:35:04.431079 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 10:35:04.431079 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 10:35:04.431992 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 10:35:04.431992 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:35:04.431992 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:35:04.431992 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:35:04.431992 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:35:04.431992 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:35:04.431992 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:35:04.431992 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:35:04.431992 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application with Lock( Jul 27 10:35:04.431992 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:35:04.431992 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:35:04.431992 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:35:04.431992 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:35:04.431992 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:35:04.431992 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:35:04.431992 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:35:04.431992 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:35:04.432877 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 10:35:04.432877 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 10:35:04.432877 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 10:35:04.432877 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 10:35:04.432877 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 10:35:04.432877 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 10:35:04.432877 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 10:35:04.432877 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Jul 27 10:35:04.432877 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 291/594] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Jul 27 10:35:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 10:35:04.433492 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c462d8b-2f40-4b52-a595-dd27e6caf153 None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:04.441618 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d74b7ee-3850-4eeb-8a3d-9c3279183669 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:04.442250 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d74b7ee-3850-4eeb-8a3d-9c3279183669 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:04.442528 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d74b7ee-3850-4eeb-8a3d-9c3279183669 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:04.443254 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d74b7ee-3850-4eeb-8a3d-9c3279183669 None admin] PATH_INFO: `/v3/domains` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:04.443592 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d74b7ee-3850-4eeb-8a3d-9c3279183669 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86827) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:35:04.443902 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d74b7ee-3850-4eeb-8a3d-9c3279183669 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:04.445330 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d74b7ee-3850-4eeb-8a3d-9c3279183669 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:04.447918 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:04.461177 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 304/595] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 10:35:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:35:04.462309 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d74b7ee-3850-4eeb-8a3d-9c3279183669 None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:04.560682 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c462d8b-2f40-4b52-a595-dd27e6caf153 None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:04.576192 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d74b7ee-3850-4eeb-8a3d-9c3279183669 None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:04.598494 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4afa90c-76f3-46e0-b1d8-2486e23a00ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:04.599177 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4afa90c-76f3-46e0-b1d8-2486e23a00ad None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:04.599462 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4afa90c-76f3-46e0-b1d8-2486e23a00ad None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:04.599738 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4afa90c-76f3-46e0-b1d8-2486e23a00ad None admin] PATH_INFO: `/v3/users/neutron` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:04.608530 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4afa90c-76f3-46e0-b1d8-2486e23a00ad None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:04.610037 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4afa90c-76f3-46e0-b1d8-2486e23a00ad None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:04.616410 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b2baa4-d681-4b1c-a79c-820a69981c3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:04.617241 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b2baa4-d681-4b1c-a79c-820a69981c3e None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:04.617674 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b2baa4-d681-4b1c-a79c-820a69981c3e None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:04.618097 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b2baa4-d681-4b1c-a79c-820a69981c3e None admin] PATH_INFO: `/v3/projects` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:04.619051 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b2baa4-d681-4b1c-a79c-820a69981c3e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86827) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:35:04.619512 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b2baa4-d681-4b1c-a79c-820a69981c3e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:04.621084 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b2baa4-d681-4b1c-a79c-820a69981c3e None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:04.622393 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application [None req-b4afa90c-76f3-46e0-b1d8-2486e23a00ad None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 27 10:35:04.622393 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:35:04.622393 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:35:04.622393 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:35:04.622393 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:35:04.622393 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:35:04.622393 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:35:04.622393 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:35:04.622393 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:35:04.622393 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:35:04.622393 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:35:04.622393 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:35:04.622393 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 10:35:04.622393 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 10:35:04.622393 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 10:35:04.622393 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 10:35:04.623527 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:35:04.623527 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:35:04.623527 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 10:35:04.623527 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 10:35:04.623527 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 10:35:04.623527 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 10:35:04.623527 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:35:04.623527 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:35:04.623527 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:35:04.623527 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:35:04.623527 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:35:04.623527 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application with Lock( Jul 27 10:35:04.623527 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:35:04.623527 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:35:04.623527 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:35:04.623527 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:35:04.623527 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:35:04.623527 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:35:04.624677 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:35:04.624677 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:35:04.624677 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 10:35:04.624677 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 10:35:04.624677 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 10:35:04.624677 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 10:35:04.624677 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 10:35:04.624677 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 10:35:04.624677 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 27 10:35:04.624677 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Jul 27 10:35:04.628447 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 292/596] 10.222.0.36 () {56 vars in 1102 bytes} [Wed Jul 27 10:35:04 2022] GET /identity/v3/users/neutron => generated 85 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 10:35:04.629293 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4afa90c-76f3-46e0-b1d8-2486e23a00ad None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:04.638324 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 305/597] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Jul 27 10:35:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:35:04.639458 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b2baa4-d681-4b1c-a79c-820a69981c3e None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:04.682312 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:04.711701 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:04.744100 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4afa90c-76f3-46e0-b1d8-2486e23a00ad None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:04.759776 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b2baa4-d681-4b1c-a79c-820a69981c3e None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:04.781563 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dea6396-795a-4f3c-b5da-b6fcb36918ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:04.782467 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dea6396-795a-4f3c-b5da-b6fcb36918ec None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:04.782920 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dea6396-795a-4f3c-b5da-b6fcb36918ec None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:04.783427 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dea6396-795a-4f3c-b5da-b6fcb36918ec None admin] PATH_INFO: `/v3/projects/service` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:04.791394 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dea6396-795a-4f3c-b5da-b6fcb36918ec None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:04.792996 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dea6396-795a-4f3c-b5da-b6fcb36918ec None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:04.794666 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb67703-ce3b-4301-81ba-16c751bd989f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:04.795366 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb67703-ce3b-4301-81ba-16c751bd989f None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:04.795720 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb67703-ce3b-4301-81ba-16c751bd989f None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:04.796073 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb67703-ce3b-4301-81ba-16c751bd989f None admin] PATH_INFO: `/v3/users` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:04.796876 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb67703-ce3b-4301-81ba-16c751bd989f None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=86827) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:35:04.797189 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb67703-ce3b-4301-81ba-16c751bd989f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:04.798622 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb67703-ce3b-4301-81ba-16c751bd989f None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:04.805854 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application [None req-5dea6396-795a-4f3c-b5da-b6fcb36918ec None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 10:35:04.805854 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:35:04.805854 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:35:04.805854 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:35:04.805854 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:35:04.805854 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:35:04.805854 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:35:04.805854 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:35:04.805854 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:35:04.805854 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:35:04.805854 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:35:04.805854 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:35:04.805854 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 10:35:04.805854 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 10:35:04.805854 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 10:35:04.806962 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 10:35:04.806962 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:35:04.806962 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:35:04.806962 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:35:04.806962 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:35:04.806962 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:35:04.806962 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:35:04.806962 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:35:04.806962 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application with Lock( Jul 27 10:35:04.806962 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:35:04.806962 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:35:04.806962 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:35:04.806962 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:35:04.806962 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:35:04.806962 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:35:04.806962 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:35:04.806962 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:35:04.812690 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 10:35:04.812690 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 10:35:04.812690 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 10:35:04.812690 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 10:35:04.812690 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 10:35:04.812690 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 10:35:04.812690 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 10:35:04.812690 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Jul 27 10:35:04.812690 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 293/598] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Jul 27 10:35:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 10:35:04.812690 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dea6396-795a-4f3c-b5da-b6fcb36918ec None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:04.828766 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 306/599] 10.222.0.36 () {56 vars in 1139 bytes} [Wed Jul 27 10:35:04 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:35:04.828766 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e6ac1a-613a-4e05-8902-b4ca7743ac5a None None] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:04.828766 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e6ac1a-613a-4e05-8902-b4ca7743ac5a None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:04.828766 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e6ac1a-613a-4e05-8902-b4ca7743ac5a None None] PATH_INFO: `/` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:04.828766 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 307/600] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 10:35:04 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 10:35:04.828766 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e6ac1a-613a-4e05-8902-b4ca7743ac5a None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:04.924474 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dea6396-795a-4f3c-b5da-b6fcb36918ec None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:04.932109 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:04.948102 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e6ac1a-613a-4e05-8902-b4ca7743ac5a None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:04.962980 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:04.970732 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4812a4eb-e4f0-4295-ad23-f45e5e1c6aa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:04.971516 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4812a4eb-e4f0-4295-ad23-f45e5e1c6aa1 None admin] REQUEST_METHOD: `PUT` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:04.972146 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4812a4eb-e4f0-4295-ad23-f45e5e1c6aa1 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:04.972551 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4812a4eb-e4f0-4295-ad23-f45e5e1c6aa1 None admin] PATH_INFO: `/v3/projects/5f296d8e3b8f4001909addc49d8dac01/users/e059d8b6c28c4fcba925d05d7c98e4ac/roles/9e0038c6bbd34556af3f84f9e2a7c2c5` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:04.987680 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4812a4eb-e4f0-4295-ad23-f45e5e1c6aa1 None admin] RBAC: Authorizing `identity:create_grant(project_id=5f296d8e3b8f4001909addc49d8dac01, user_id=e059d8b6c28c4fcba925d05d7c98e4ac, role_id=9e0038c6bbd34556af3f84f9e2a7c2c5)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:04.988046 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520a69ac-0eff-462c-93d9-7c1c5bf8120d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:04.988891 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520a69ac-0eff-462c-93d9-7c1c5bf8120d None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:04.989126 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4812a4eb-e4f0-4295-ad23-f45e5e1c6aa1 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:04.989496 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520a69ac-0eff-462c-93d9-7c1c5bf8120d None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:04.989990 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520a69ac-0eff-462c-93d9-7c1c5bf8120d None admin] PATH_INFO: `/v3/projects` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:04.990936 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520a69ac-0eff-462c-93d9-7c1c5bf8120d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86827) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:35:04.991581 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520a69ac-0eff-462c-93d9-7c1c5bf8120d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:04.993797 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520a69ac-0eff-462c-93d9-7c1c5bf8120d None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:05.011513 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 294/601] 10.222.0.36 () {58 vars in 1439 bytes} [Wed Jul 27 10:35:04 2022] PUT /identity/v3/projects/5f296d8e3b8f4001909addc49d8dac01/users/e059d8b6c28c4fcba925d05d7c98e4ac/roles/9e0038c6bbd34556af3f84f9e2a7c2c5 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:35:05.011513 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4812a4eb-e4f0-4295-ad23-f45e5e1c6aa1 None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:05.011777 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 308/602] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Jul 27 10:35:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:35:05.011777 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdb7d72-99e4-4d92-b602-626537ab59cf None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:05.011777 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdb7d72-99e4-4d92-b602-626537ab59cf None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:05.011777 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdb7d72-99e4-4d92-b602-626537ab59cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:05.030355 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-6fdb7d72-99e4-4d92-b602-626537ab59cf None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:35:05.031736 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-6fdb7d72-99e4-4d92-b602-626537ab59cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:35:05.115136 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 309/603] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 10:35:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 10:35:05.116663 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdb7d72-99e4-4d92-b602-626537ab59cf None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:05.134780 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4812a4eb-e4f0-4295-ad23-f45e5e1c6aa1 None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:05.173537 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1128ea5b-831b-40d3-9857-c31000de6fd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:05.174449 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1128ea5b-831b-40d3-9857-c31000de6fd8 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:05.174951 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1128ea5b-831b-40d3-9857-c31000de6fd8 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:05.175328 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1128ea5b-831b-40d3-9857-c31000de6fd8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:05.186816 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1128ea5b-831b-40d3-9857-c31000de6fd8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:05.188367 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1128ea5b-831b-40d3-9857-c31000de6fd8 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:05.205004 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application [None req-1128ea5b-831b-40d3-9857-c31000de6fd8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 10:35:05.205004 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:35:05.205004 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:35:05.205004 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:35:05.205004 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:35:05.205004 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:35:05.205004 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:35:05.205004 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:35:05.205004 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:35:05.205004 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:35:05.205004 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:35:05.205004 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:35:05.205004 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 10:35:05.205004 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 10:35:05.205004 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 10:35:05.206137 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 10:35:05.206137 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:35:05.206137 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:35:05.206137 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:35:05.206137 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:35:05.206137 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:35:05.206137 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:35:05.206137 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:35:05.206137 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application with Lock( Jul 27 10:35:05.206137 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:35:05.206137 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:35:05.206137 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:35:05.206137 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:35:05.206137 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:35:05.206137 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:35:05.206137 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:35:05.206137 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:35:05.207207 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 10:35:05.207207 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 10:35:05.207207 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 10:35:05.207207 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Jul 27 10:35:05.207437 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 295/604] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 10:35:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 10:35:05.208063 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1128ea5b-831b-40d3-9857-c31000de6fd8 None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:05.222688 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:05.264713 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdb7d72-99e4-4d92-b602-626537ab59cf None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:05.268203 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:05.312220 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a371b2fb-d452-49f9-8a74-03dc56bc0f14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:05.312935 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a371b2fb-d452-49f9-8a74-03dc56bc0f14 None admin] REQUEST_METHOD: `PUT` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:05.313487 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a371b2fb-d452-49f9-8a74-03dc56bc0f14 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:05.313773 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a371b2fb-d452-49f9-8a74-03dc56bc0f14 None admin] PATH_INFO: `/v3/projects/5f296d8e3b8f4001909addc49d8dac01/users/864d35f268cf45148756109759bb2743/roles/9e0038c6bbd34556af3f84f9e2a7c2c5` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:05.334497 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1128ea5b-831b-40d3-9857-c31000de6fd8 None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:05.335687 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a371b2fb-d452-49f9-8a74-03dc56bc0f14 None admin] RBAC: Authorizing `identity:create_grant(project_id=5f296d8e3b8f4001909addc49d8dac01, user_id=864d35f268cf45148756109759bb2743, role_id=9e0038c6bbd34556af3f84f9e2a7c2c5)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:05.338304 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a371b2fb-d452-49f9-8a74-03dc56bc0f14 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:05.354441 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 310/605] 10.222.0.36 () {58 vars in 1439 bytes} [Wed Jul 27 10:35:05 2022] PUT /identity/v3/projects/5f296d8e3b8f4001909addc49d8dac01/users/864d35f268cf45148756109759bb2743/roles/9e0038c6bbd34556af3f84f9e2a7c2c5 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:35:05.363040 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a371b2fb-d452-49f9-8a74-03dc56bc0f14 None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:05.377249 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31c09bd-d200-440d-920f-5015a07e625d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:05.378009 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31c09bd-d200-440d-920f-5015a07e625d None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:05.378462 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31c09bd-d200-440d-920f-5015a07e625d None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:05.378889 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31c09bd-d200-440d-920f-5015a07e625d None admin] PATH_INFO: `/v3/roles/service` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:05.395445 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31c09bd-d200-440d-920f-5015a07e625d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 10:35:05.402754 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application [None req-b31c09bd-d200-440d-920f-5015a07e625d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 27 10:35:05.402754 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:35:05.402754 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:35:05.402754 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:35:05.402754 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:35:05.402754 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:35:05.402754 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:35:05.402754 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:35:05.402754 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:35:05.402754 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:35:05.402754 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:35:05.402754 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:35:05.402754 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 10:35:05.402754 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 10:35:05.402754 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 10:35:05.402754 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 10:35:05.403981 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 10:35:05.403981 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 10:35:05.403981 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 10:35:05.403981 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 10:35:05.403981 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:35:05.403981 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:35:05.403981 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:35:05.403981 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:35:05.403981 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:35:05.403981 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:35:05.403981 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:35:05.403981 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application with Lock( Jul 27 10:35:05.403981 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:35:05.403981 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:35:05.403981 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:35:05.403981 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:35:05.405018 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:35:05.405018 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:35:05.405018 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:35:05.405018 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:35:05.405018 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 10:35:05.405018 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 10:35:05.405018 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 10:35:05.405018 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 10:35:05.405018 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 10:35:05.405018 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 10:35:05.405018 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 27 10:35:05.405018 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Jul 27 10:35:05.407341 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 296/606] 10.222.0.36 () {56 vars in 1102 bytes} [Wed Jul 27 10:35:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 10:35:05.413668 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31c09bd-d200-440d-920f-5015a07e625d None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:05.496747 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:05.539403 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31c09bd-d200-440d-920f-5015a07e625d None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:05.570812 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a371b2fb-d452-49f9-8a74-03dc56bc0f14 None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:05.588872 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7935610-896d-4d49-94bc-70de92433013 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:05.590161 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7935610-896d-4d49-94bc-70de92433013 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:05.590556 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7935610-896d-4d49-94bc-70de92433013 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:05.590986 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7935610-896d-4d49-94bc-70de92433013 None admin] PATH_INFO: `/v3/roles` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:05.592111 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7935610-896d-4d49-94bc-70de92433013 None admin] RBAC: Adding query filter params (name=service) {{(pid=86828) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:35:05.592522 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7935610-896d-4d49-94bc-70de92433013 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:05.594462 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7935610-896d-4d49-94bc-70de92433013 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:05.605366 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 297/607] 10.222.0.36 () {56 vars in 1103 bytes} [Wed Jul 27 10:35:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 10:35:05.610906 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7935610-896d-4d49-94bc-70de92433013 None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:05.614553 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463480ba-ada8-4c4d-95b0-6f43e6d64f0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:05.615282 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463480ba-ada8-4c4d-95b0-6f43e6d64f0e None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:05.615638 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463480ba-ada8-4c4d-95b0-6f43e6d64f0e None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:05.616009 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463480ba-ada8-4c4d-95b0-6f43e6d64f0e None admin] PATH_INFO: `/v3/domains` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:05.617317 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463480ba-ada8-4c4d-95b0-6f43e6d64f0e None admin] RBAC: Adding query filter params (name=Default) {{(pid=86827) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:35:05.618081 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463480ba-ada8-4c4d-95b0-6f43e6d64f0e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:05.619247 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463480ba-ada8-4c4d-95b0-6f43e6d64f0e None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:05.632053 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 311/608] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 10:35:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:35:05.639352 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463480ba-ada8-4c4d-95b0-6f43e6d64f0e None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:05.660732 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:05.763628 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7935610-896d-4d49-94bc-70de92433013 None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:05.785111 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463480ba-ada8-4c4d-95b0-6f43e6d64f0e None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:05.798172 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db456700-cfef-402b-9286-bc21b7fde9e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:05.799047 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db456700-cfef-402b-9286-bc21b7fde9e5 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:05.799623 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db456700-cfef-402b-9286-bc21b7fde9e5 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:05.800051 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db456700-cfef-402b-9286-bc21b7fde9e5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:05.810855 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:05.821765 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902fe014-93f2-490a-91f4-4f346ee02829 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:05.822254 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db456700-cfef-402b-9286-bc21b7fde9e5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:05.823028 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902fe014-93f2-490a-91f4-4f346ee02829 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:05.823028 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902fe014-93f2-490a-91f4-4f346ee02829 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:05.823028 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902fe014-93f2-490a-91f4-4f346ee02829 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:05.824888 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db456700-cfef-402b-9286-bc21b7fde9e5 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:05.830305 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902fe014-93f2-490a-91f4-4f346ee02829 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:05.831774 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902fe014-93f2-490a-91f4-4f346ee02829 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:05.844370 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application [None req-902fe014-93f2-490a-91f4-4f346ee02829 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 10:35:05.844370 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:35:05.844370 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:35:05.844370 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:35:05.844370 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:35:05.844370 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:35:05.844370 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:35:05.844370 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:35:05.844370 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:35:05.844370 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:35:05.844370 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:35:05.844370 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:35:05.844370 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 10:35:05.844370 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 10:35:05.844370 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 10:35:05.845536 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 10:35:05.845536 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:35:05.845536 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:35:05.845536 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:35:05.845536 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:35:05.845536 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:35:05.845536 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:35:05.845536 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:35:05.845536 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application with Lock( Jul 27 10:35:05.845536 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:35:05.845536 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:35:05.845536 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:35:05.845536 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:35:05.845536 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:35:05.845536 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:35:05.845536 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:35:05.845536 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:35:05.846791 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 10:35:05.846791 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 10:35:05.846791 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 10:35:05.846791 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 10:35:05.846791 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 10:35:05.846791 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 10:35:05.846791 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 10:35:05.846791 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Jul 27 10:35:05.846791 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 312/609] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Jul 27 10:35:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 10:35:05.858627 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902fe014-93f2-490a-91f4-4f346ee02829 None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:05.860321 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application [None req-db456700-cfef-402b-9286-bc21b7fde9e5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 10:35:05.860321 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:35:05.860321 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:35:05.860321 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:35:05.860321 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:35:05.860321 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:35:05.860321 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:35:05.860321 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:35:05.860321 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:35:05.860321 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:35:05.860321 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:35:05.860321 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:35:05.860321 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 10:35:05.860321 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 10:35:05.860321 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 10:35:05.861369 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 10:35:05.861369 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:35:05.861369 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:35:05.861369 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:35:05.861369 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:35:05.861369 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:35:05.861369 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:35:05.861369 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:35:05.861369 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application with Lock( Jul 27 10:35:05.861369 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:35:05.861369 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:35:05.861369 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:35:05.861369 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:35:05.861369 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:35:05.861369 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:35:05.861369 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:35:05.861369 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:35:05.862467 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 10:35:05.862467 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 10:35:05.862467 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 10:35:05.862467 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Jul 27 10:35:05.864351 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 298/610] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 10:35:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 10:35:05.869550 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db456700-cfef-402b-9286-bc21b7fde9e5 None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:05.935432 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:05.967408 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902fe014-93f2-490a-91f4-4f346ee02829 None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:06.023405 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3161a5f-fd9f-4cae-a329-6ed2bd881435 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:06.025096 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3161a5f-fd9f-4cae-a329-6ed2bd881435 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:06.025853 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3161a5f-fd9f-4cae-a329-6ed2bd881435 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:06.026522 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3161a5f-fd9f-4cae-a329-6ed2bd881435 None admin] PATH_INFO: `/v3/projects` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:06.028100 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3161a5f-fd9f-4cae-a329-6ed2bd881435 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86827) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:35:06.028824 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3161a5f-fd9f-4cae-a329-6ed2bd881435 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:06.039782 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3161a5f-fd9f-4cae-a329-6ed2bd881435 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:06.040887 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db456700-cfef-402b-9286-bc21b7fde9e5 None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:06.070922 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 313/611] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Jul 27 10:35:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:35:06.076487 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5338ab86-3bbf-497c-a9a5-f04a61d9a00d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:06.076923 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3161a5f-fd9f-4cae-a329-6ed2bd881435 None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:06.077109 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5338ab86-3bbf-497c-a9a5-f04a61d9a00d None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:06.077387 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5338ab86-3bbf-497c-a9a5-f04a61d9a00d None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:06.077659 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5338ab86-3bbf-497c-a9a5-f04a61d9a00d None admin] PATH_INFO: `/v3/domains` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:06.078357 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5338ab86-3bbf-497c-a9a5-f04a61d9a00d None admin] RBAC: Adding query filter params (name=Default) {{(pid=86828) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:35:06.078700 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5338ab86-3bbf-497c-a9a5-f04a61d9a00d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:06.080062 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5338ab86-3bbf-497c-a9a5-f04a61d9a00d None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:06.093430 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 299/612] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 10:35:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:35:06.103681 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5338ab86-3bbf-497c-a9a5-f04a61d9a00d None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:06.156785 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:06.236404 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3161a5f-fd9f-4cae-a329-6ed2bd881435 None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:06.280227 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5338ab86-3bbf-497c-a9a5-f04a61d9a00d None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:06.324820 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b058521-e93e-413a-b5ef-e1ab57ffe173 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:06.325486 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b058521-e93e-413a-b5ef-e1ab57ffe173 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:06.325769 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b058521-e93e-413a-b5ef-e1ab57ffe173 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:06.326046 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b058521-e93e-413a-b5ef-e1ab57ffe173 None admin] PATH_INFO: `/v3/users/swift` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:06.332180 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270211a3-1d22-4bd0-929c-89a278f53ac6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:06.333112 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270211a3-1d22-4bd0-929c-89a278f53ac6 None admin] REQUEST_METHOD: `PUT` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:06.333579 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270211a3-1d22-4bd0-929c-89a278f53ac6 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:06.333822 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b058521-e93e-413a-b5ef-e1ab57ffe173 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:06.334223 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270211a3-1d22-4bd0-929c-89a278f53ac6 None admin] PATH_INFO: `/v3/projects/5f296d8e3b8f4001909addc49d8dac01/users/c7922d0a1a7c4e808d0b2a80dca0da40/roles/9e0038c6bbd34556af3f84f9e2a7c2c5` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:06.338624 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b058521-e93e-413a-b5ef-e1ab57ffe173 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:06.344939 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:06.349696 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application [None req-9b058521-e93e-413a-b5ef-e1ab57ffe173 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 27 10:35:06.349696 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:35:06.349696 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:35:06.349696 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:35:06.349696 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:35:06.349696 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:35:06.349696 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:35:06.349696 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:35:06.349696 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:35:06.349696 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:35:06.349696 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:35:06.349696 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:35:06.349696 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 10:35:06.349696 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 10:35:06.349696 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 10:35:06.349696 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 10:35:06.351216 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:35:06.351216 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:35:06.351216 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 10:35:06.351216 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 10:35:06.351216 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 10:35:06.351216 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 10:35:06.351216 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:35:06.351216 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:35:06.351216 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:35:06.351216 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:35:06.351216 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:35:06.351216 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application with Lock( Jul 27 10:35:06.351216 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:35:06.351216 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:35:06.351216 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:35:06.351216 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:35:06.351216 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:35:06.351216 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:35:06.352755 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:35:06.352755 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:35:06.352755 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 10:35:06.352755 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 10:35:06.352755 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 10:35:06.352755 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 10:35:06.352755 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 10:35:06.352755 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 10:35:06.352755 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 27 10:35:06.352755 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Jul 27 10:35:06.352755 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 300/613] 10.222.0.36 () {56 vars in 1096 bytes} [Wed Jul 27 10:35:06 2022] GET /identity/v3/users/swift => generated 83 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 10:35:06.356689 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b058521-e93e-413a-b5ef-e1ab57ffe173 None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:06.372201 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270211a3-1d22-4bd0-929c-89a278f53ac6 None admin] RBAC: Authorizing `identity:create_grant(project_id=5f296d8e3b8f4001909addc49d8dac01, user_id=c7922d0a1a7c4e808d0b2a80dca0da40, role_id=9e0038c6bbd34556af3f84f9e2a7c2c5)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:06.375315 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270211a3-1d22-4bd0-929c-89a278f53ac6 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:06.401091 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 314/614] 10.222.0.36 () {58 vars in 1439 bytes} [Wed Jul 27 10:35:06 2022] PUT /identity/v3/projects/5f296d8e3b8f4001909addc49d8dac01/users/c7922d0a1a7c4e808d0b2a80dca0da40/roles/9e0038c6bbd34556af3f84f9e2a7c2c5 => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:35:06.506361 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967db1aa-04d6-4a33-b140-8ac87b08accc None None] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:06.507233 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967db1aa-04d6-4a33-b140-8ac87b08accc None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:06.507594 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967db1aa-04d6-4a33-b140-8ac87b08accc None None] PATH_INFO: `/` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:06.509375 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 315/615] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 10:35:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 10:35:06.515018 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b058521-e93e-413a-b5ef-e1ab57ffe173 None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:06.516670 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1de2caa-b8db-4642-ab6c-ac539d4f7e2d None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:06.517359 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:06.519651 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1de2caa-b8db-4642-ab6c-ac539d4f7e2d None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:06.521176 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1de2caa-b8db-4642-ab6c-ac539d4f7e2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:06.557521 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-c1de2caa-b8db-4642-ab6c-ac539d4f7e2d None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:35:06.559116 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-c1de2caa-b8db-4642-ab6c-ac539d4f7e2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:35:06.593631 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf012cb0-be54-4c3e-8382-0dc9d6ddaa20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:06.594590 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf012cb0-be54-4c3e-8382-0dc9d6ddaa20 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:06.595907 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf012cb0-be54-4c3e-8382-0dc9d6ddaa20 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:06.596346 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf012cb0-be54-4c3e-8382-0dc9d6ddaa20 None admin] PATH_INFO: `/v3/users` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:06.598687 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf012cb0-be54-4c3e-8382-0dc9d6ddaa20 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=86828) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:35:06.599196 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf012cb0-be54-4c3e-8382-0dc9d6ddaa20 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:06.601663 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf012cb0-be54-4c3e-8382-0dc9d6ddaa20 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:06.622871 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 301/616] 10.222.0.36 () {56 vars in 1135 bytes} [Wed Jul 27 10:35:06 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:35:06.628763 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf012cb0-be54-4c3e-8382-0dc9d6ddaa20 None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:06.679457 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 316/617] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 10:35:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 27 10:35:06.706436 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1de2caa-b8db-4642-ab6c-ac539d4f7e2d None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:06.772154 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf012cb0-be54-4c3e-8382-0dc9d6ddaa20 None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:06.810975 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6645f067-f3e5-4975-bcfa-27c3928873b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:06.811636 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6645f067-f3e5-4975-bcfa-27c3928873b3 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:06.811925 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6645f067-f3e5-4975-bcfa-27c3928873b3 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:06.812206 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6645f067-f3e5-4975-bcfa-27c3928873b3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:06.834985 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6645f067-f3e5-4975-bcfa-27c3928873b3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:06.836481 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6645f067-f3e5-4975-bcfa-27c3928873b3 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:06.862862 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application [None req-6645f067-f3e5-4975-bcfa-27c3928873b3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 10:35:06.862862 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:35:06.862862 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:35:06.862862 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:35:06.862862 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:35:06.862862 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:35:06.862862 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:35:06.862862 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:35:06.862862 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:35:06.862862 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:35:06.862862 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:35:06.862862 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:35:06.862862 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 10:35:06.862862 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 10:35:06.862862 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 10:35:06.863786 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 10:35:06.863786 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:35:06.863786 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:35:06.863786 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:35:06.863786 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:35:06.863786 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:35:06.863786 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:35:06.863786 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:35:06.863786 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application with Lock( Jul 27 10:35:06.863786 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:35:06.863786 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:35:06.863786 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:35:06.863786 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:35:06.863786 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:35:06.863786 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:35:06.863786 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:35:06.863786 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:35:06.864675 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 10:35:06.864675 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 10:35:06.864675 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 10:35:06.864675 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Jul 27 10:35:06.864675 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 302/618] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 10:35:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 10:35:06.870252 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6645f067-f3e5-4975-bcfa-27c3928873b3 None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:06.888382 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:06.938358 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1de2caa-b8db-4642-ab6c-ac539d4f7e2d None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:06.971644 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:06.984084 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6645f067-f3e5-4975-bcfa-27c3928873b3 None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:07.017792 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9655bf26-f2b0-4eb1-abcc-8c69103a33fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:07.018903 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9655bf26-f2b0-4eb1-abcc-8c69103a33fa None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:07.019395 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9655bf26-f2b0-4eb1-abcc-8c69103a33fa None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:07.019949 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9655bf26-f2b0-4eb1-abcc-8c69103a33fa None admin] PATH_INFO: `/v3/roles/service` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:07.020202 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a309b583-58cc-4390-9d5f-bcc5731bea88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:07.020825 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a309b583-58cc-4390-9d5f-bcc5731bea88 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:07.021115 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a309b583-58cc-4390-9d5f-bcc5731bea88 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:07.022686 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a309b583-58cc-4390-9d5f-bcc5731bea88 None admin] PATH_INFO: `/v3/domains` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:07.022686 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a309b583-58cc-4390-9d5f-bcc5731bea88 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86828) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:35:07.022686 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a309b583-58cc-4390-9d5f-bcc5731bea88 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:07.023992 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a309b583-58cc-4390-9d5f-bcc5731bea88 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:07.036561 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 303/619] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 10:35:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:35:07.042710 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a309b583-58cc-4390-9d5f-bcc5731bea88 None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:07.046953 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9655bf26-f2b0-4eb1-abcc-8c69103a33fa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 10:35:07.051920 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application [None req-9655bf26-f2b0-4eb1-abcc-8c69103a33fa None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 27 10:35:07.051920 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:35:07.051920 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:35:07.051920 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:35:07.051920 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:35:07.051920 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:35:07.051920 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:35:07.051920 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:35:07.051920 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:35:07.051920 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:35:07.051920 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:35:07.051920 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:35:07.051920 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 10:35:07.051920 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 10:35:07.051920 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 10:35:07.051920 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 10:35:07.052990 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 10:35:07.052990 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 10:35:07.052990 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 10:35:07.052990 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 10:35:07.052990 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:35:07.052990 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:35:07.052990 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:35:07.052990 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:35:07.052990 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:35:07.052990 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:35:07.052990 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:35:07.052990 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application with Lock( Jul 27 10:35:07.052990 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:35:07.052990 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:35:07.052990 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:35:07.052990 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:35:07.054436 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:35:07.054436 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:35:07.054436 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:35:07.054436 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:35:07.054436 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 10:35:07.054436 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 10:35:07.054436 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 10:35:07.054436 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 10:35:07.054436 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 10:35:07.054436 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 10:35:07.054436 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 27 10:35:07.054436 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Jul 27 10:35:07.054436 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 317/620] 10.222.0.36 () {56 vars in 1102 bytes} [Wed Jul 27 10:35:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 347 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 10:35:07.059803 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9655bf26-f2b0-4eb1-abcc-8c69103a33fa None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:07.115423 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:07.147536 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a309b583-58cc-4390-9d5f-bcc5731bea88 None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:07.221220 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0529004-f4f7-47d8-a69b-78dc093cfb36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:07.222458 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0529004-f4f7-47d8-a69b-78dc093cfb36 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:07.224381 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0529004-f4f7-47d8-a69b-78dc093cfb36 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:07.224878 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0529004-f4f7-47d8-a69b-78dc093cfb36 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:07.238963 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0529004-f4f7-47d8-a69b-78dc093cfb36 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:07.242378 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0529004-f4f7-47d8-a69b-78dc093cfb36 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:07.261381 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application [None req-f0529004-f4f7-47d8-a69b-78dc093cfb36 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 10:35:07.261381 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:35:07.261381 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:35:07.261381 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:35:07.261381 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:35:07.261381 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:35:07.261381 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:35:07.261381 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:35:07.261381 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:35:07.261381 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:35:07.261381 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:35:07.261381 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:35:07.261381 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 10:35:07.261381 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 10:35:07.261381 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 10:35:07.262312 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 10:35:07.262312 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:35:07.262312 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:35:07.262312 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:35:07.262312 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:35:07.262312 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:35:07.262312 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:35:07.262312 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:35:07.262312 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application with Lock( Jul 27 10:35:07.262312 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:35:07.262312 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:35:07.262312 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:35:07.262312 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:35:07.262312 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:35:07.262312 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:35:07.262312 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:35:07.262312 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:35:07.263245 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 10:35:07.263245 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 10:35:07.263245 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 10:35:07.263245 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 10:35:07.263245 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 10:35:07.263245 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 10:35:07.263245 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 10:35:07.263245 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Jul 27 10:35:07.267891 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9655bf26-f2b0-4eb1-abcc-8c69103a33fa None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:07.271896 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 304/621] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Jul 27 10:35:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 10:35:07.277477 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0529004-f4f7-47d8-a69b-78dc093cfb36 None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:07.329329 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c8b1d7-928f-4a4a-bfe3-26817cfecb8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:07.330013 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c8b1d7-928f-4a4a-bfe3-26817cfecb8c None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:07.330230 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c8b1d7-928f-4a4a-bfe3-26817cfecb8c None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:07.330507 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c8b1d7-928f-4a4a-bfe3-26817cfecb8c None admin] PATH_INFO: `/v3/roles` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:07.331270 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c8b1d7-928f-4a4a-bfe3-26817cfecb8c None admin] RBAC: Adding query filter params (name=service) {{(pid=86827) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:35:07.332258 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c8b1d7-928f-4a4a-bfe3-26817cfecb8c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:07.332942 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c8b1d7-928f-4a4a-bfe3-26817cfecb8c None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:07.341531 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 318/622] 10.222.0.36 () {56 vars in 1103 bytes} [Wed Jul 27 10:35:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 10:35:07.346943 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c8b1d7-928f-4a4a-bfe3-26817cfecb8c None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:07.383032 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:07.401130 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0529004-f4f7-47d8-a69b-78dc093cfb36 None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:07.436975 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20051f2-aa13-4b60-b782-da8cb6332f61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:07.438005 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20051f2-aa13-4b60-b782-da8cb6332f61 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:07.438494 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20051f2-aa13-4b60-b782-da8cb6332f61 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:07.439129 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20051f2-aa13-4b60-b782-da8cb6332f61 None admin] PATH_INFO: `/v3/projects` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:07.440097 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20051f2-aa13-4b60-b782-da8cb6332f61 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86828) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:35:07.440612 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20051f2-aa13-4b60-b782-da8cb6332f61 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:07.446806 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:07.447573 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20051f2-aa13-4b60-b782-da8cb6332f61 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:07.462392 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 305/623] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Jul 27 10:35:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:35:07.470707 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c8b1d7-928f-4a4a-bfe3-26817cfecb8c None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:07.471856 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20051f2-aa13-4b60-b782-da8cb6332f61 None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:07.514669 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c14bfd-4752-4381-a5eb-9f98cf2e4a32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:07.515344 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c14bfd-4752-4381-a5eb-9f98cf2e4a32 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:07.515626 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c14bfd-4752-4381-a5eb-9f98cf2e4a32 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:07.515903 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c14bfd-4752-4381-a5eb-9f98cf2e4a32 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:07.543945 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c14bfd-4752-4381-a5eb-9f98cf2e4a32 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:07.545378 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c14bfd-4752-4381-a5eb-9f98cf2e4a32 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:07.564448 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application [None req-78c14bfd-4752-4381-a5eb-9f98cf2e4a32 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 10:35:07.564448 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:35:07.564448 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:35:07.564448 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:35:07.564448 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:35:07.564448 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:35:07.564448 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:35:07.564448 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:35:07.564448 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:35:07.564448 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:35:07.564448 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:35:07.564448 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:35:07.564448 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 10:35:07.564448 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 10:35:07.564448 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 10:35:07.565364 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 10:35:07.565364 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:35:07.565364 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:35:07.565364 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:35:07.565364 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:35:07.565364 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:35:07.565364 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:35:07.565364 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:35:07.565364 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application with Lock( Jul 27 10:35:07.565364 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:35:07.565364 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:35:07.565364 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:35:07.565364 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:35:07.565364 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:35:07.565364 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:35:07.565364 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:35:07.565364 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:35:07.566247 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 10:35:07.566247 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 10:35:07.566247 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 10:35:07.566247 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Jul 27 10:35:07.566247 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 319/624] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 10:35:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 10:35:07.571450 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b976adf6-15b6-469d-baec-aaf9f228ed12 None None] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:07.571741 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b976adf6-15b6-469d-baec-aaf9f228ed12 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:07.572023 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b976adf6-15b6-469d-baec-aaf9f228ed12 None None] PATH_INFO: `/` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:07.573555 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 320/625] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 10:35:07 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 10:35:07.574271 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b976adf6-15b6-469d-baec-aaf9f228ed12 None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:07.617753 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20051f2-aa13-4b60-b782-da8cb6332f61 None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:07.653609 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513dec98-ff53-49a4-bd6a-a773f6300fee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:07.654631 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513dec98-ff53-49a4-bd6a-a773f6300fee None admin] REQUEST_METHOD: `PUT` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:07.655123 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513dec98-ff53-49a4-bd6a-a773f6300fee None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:07.655603 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513dec98-ff53-49a4-bd6a-a773f6300fee None admin] PATH_INFO: `/v3/projects/5f296d8e3b8f4001909addc49d8dac01/users/89ac506eefcd4e42a84b44fa15fb3975/roles/9e0038c6bbd34556af3f84f9e2a7c2c5` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:07.667839 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:07.671920 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513dec98-ff53-49a4-bd6a-a773f6300fee None admin] RBAC: Authorizing `identity:create_grant(project_id=5f296d8e3b8f4001909addc49d8dac01, user_id=89ac506eefcd4e42a84b44fa15fb3975, role_id=9e0038c6bbd34556af3f84f9e2a7c2c5)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:07.673638 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513dec98-ff53-49a4-bd6a-a773f6300fee None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:07.683715 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b976adf6-15b6-469d-baec-aaf9f228ed12 None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:07.689344 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 306/626] 10.222.0.36 () {58 vars in 1439 bytes} [Wed Jul 27 10:35:07 2022] PUT /identity/v3/projects/5f296d8e3b8f4001909addc49d8dac01/users/89ac506eefcd4e42a84b44fa15fb3975/roles/9e0038c6bbd34556af3f84f9e2a7c2c5 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:35:07.691168 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d41494c-545c-43f6-bfc0-e61859da5b1a None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:07.691723 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d41494c-545c-43f6-bfc0-e61859da5b1a None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:07.692209 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d41494c-545c-43f6-bfc0-e61859da5b1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:07.719168 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-1d41494c-545c-43f6-bfc0-e61859da5b1a None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:35:07.720770 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-1d41494c-545c-43f6-bfc0-e61859da5b1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:35:07.723178 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d47dd5-e0d7-4d64-9de3-b66950d2182b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:07.723906 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d47dd5-e0d7-4d64-9de3-b66950d2182b None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:07.724143 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d47dd5-e0d7-4d64-9de3-b66950d2182b None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:07.724419 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d47dd5-e0d7-4d64-9de3-b66950d2182b None admin] PATH_INFO: `/v3/domains` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:07.725247 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d47dd5-e0d7-4d64-9de3-b66950d2182b None admin] RBAC: Adding query filter params (name=Default) {{(pid=86827) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:35:07.725594 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d47dd5-e0d7-4d64-9de3-b66950d2182b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:07.727143 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d47dd5-e0d7-4d64-9de3-b66950d2182b None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:07.742044 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 321/627] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 10:35:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:35:07.747457 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d47dd5-e0d7-4d64-9de3-b66950d2182b None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:07.771116 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:07.831634 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 307/628] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 10:35:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 10:35:07.868653 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d41494c-545c-43f6-bfc0-e61859da5b1a None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:07.922998 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d47dd5-e0d7-4d64-9de3-b66950d2182b None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:07.974700 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e0be54-8a22-4f74-9069-7d5bb25fd45c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:07.975985 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e0be54-8a22-4f74-9069-7d5bb25fd45c None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:07.977342 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e0be54-8a22-4f74-9069-7d5bb25fd45c None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:07.977811 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e0be54-8a22-4f74-9069-7d5bb25fd45c None admin] PATH_INFO: `/v3/users/glance` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:07.985869 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e0be54-8a22-4f74-9069-7d5bb25fd45c None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:07.987373 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e0be54-8a22-4f74-9069-7d5bb25fd45c None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:08.000520 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application [None req-20e0be54-8a22-4f74-9069-7d5bb25fd45c None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 27 10:35:08.000520 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:35:08.000520 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:35:08.000520 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:35:08.000520 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:35:08.000520 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:35:08.000520 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:35:08.000520 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:35:08.000520 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:35:08.000520 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:35:08.000520 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:35:08.000520 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:35:08.000520 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 10:35:08.000520 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 10:35:08.000520 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 10:35:08.000520 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 10:35:08.001519 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:35:08.001519 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:35:08.001519 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 10:35:08.001519 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 10:35:08.001519 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 10:35:08.001519 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 10:35:08.001519 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:35:08.001519 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:35:08.001519 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:35:08.001519 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:35:08.001519 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:35:08.001519 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application with Lock( Jul 27 10:35:08.001519 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:35:08.001519 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:35:08.001519 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:35:08.001519 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:35:08.001519 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:35:08.001519 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:35:08.002557 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:35:08.002557 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:35:08.002557 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 10:35:08.002557 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 10:35:08.002557 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 10:35:08.002557 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 10:35:08.002557 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 10:35:08.002557 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 10:35:08.002557 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 27 10:35:08.002557 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Jul 27 10:35:08.002557 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 322/629] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jul 27 10:35:07 2022] GET /identity/v3/users/glance => generated 84 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 10:35:08.003194 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87904c9b-206a-4d4e-a6f2-29dd741e1e25 None None] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:08.003450 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87904c9b-206a-4d4e-a6f2-29dd741e1e25 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:08.003676 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87904c9b-206a-4d4e-a6f2-29dd741e1e25 None None] PATH_INFO: `/` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:08.010558 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 323/630] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 10:35:08 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 10:35:08.011295 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87904c9b-206a-4d4e-a6f2-29dd741e1e25 None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:08.013675 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d41494c-545c-43f6-bfc0-e61859da5b1a None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:08.035517 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:08.051788 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594d7360-7c43-49b5-b87d-7f82063dffbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:08.052433 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594d7360-7c43-49b5-b87d-7f82063dffbe None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:08.052713 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594d7360-7c43-49b5-b87d-7f82063dffbe None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:08.052983 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594d7360-7c43-49b5-b87d-7f82063dffbe None admin] PATH_INFO: `/v3/roles/service` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:08.065115 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594d7360-7c43-49b5-b87d-7f82063dffbe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 10:35:08.071180 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application [None req-594d7360-7c43-49b5-b87d-7f82063dffbe None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 27 10:35:08.071180 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:35:08.071180 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:35:08.071180 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:35:08.071180 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:35:08.071180 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:35:08.071180 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:35:08.071180 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:35:08.071180 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:35:08.071180 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:35:08.071180 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:35:08.071180 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:35:08.071180 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 10:35:08.071180 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 10:35:08.071180 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 10:35:08.071180 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 10:35:08.072216 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 10:35:08.072216 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 10:35:08.072216 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 10:35:08.072216 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 10:35:08.072216 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:35:08.072216 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:35:08.072216 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:35:08.072216 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:35:08.072216 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:35:08.072216 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:35:08.072216 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:35:08.072216 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application with Lock( Jul 27 10:35:08.072216 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:35:08.072216 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:35:08.072216 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:35:08.072216 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:35:08.073038 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:35:08.073038 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:35:08.073038 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:35:08.073038 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:35:08.073038 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 10:35:08.073038 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 10:35:08.073038 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 10:35:08.073038 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 10:35:08.073038 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 10:35:08.073038 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 10:35:08.073038 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 27 10:35:08.073038 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Jul 27 10:35:08.073038 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 308/631] 10.222.0.36 () {56 vars in 1102 bytes} [Wed Jul 27 10:35:07 2022] GET /identity/v3/roles/service => generated 85 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 10:35:08.074344 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca3b1ac-71fc-4ca7-aa61-658bd1414729 None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:08.074344 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca3b1ac-71fc-4ca7-aa61-658bd1414729 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:08.074502 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca3b1ac-71fc-4ca7-aa61-658bd1414729 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:08.094348 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-eca3b1ac-71fc-4ca7-aa61-658bd1414729 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:35:08.095972 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-eca3b1ac-71fc-4ca7-aa61-658bd1414729 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:35:08.123561 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:08.131721 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87904c9b-206a-4d4e-a6f2-29dd741e1e25 None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:08.167861 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7efcad0-48e2-4dc9-b634-f2308e36d267 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:08.168802 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7efcad0-48e2-4dc9-b634-f2308e36d267 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:08.169269 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7efcad0-48e2-4dc9-b634-f2308e36d267 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:08.169701 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7efcad0-48e2-4dc9-b634-f2308e36d267 None admin] PATH_INFO: `/v3/users` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:08.170029 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 309/632] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 10:35:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 10:35:08.171021 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca3b1ac-71fc-4ca7-aa61-658bd1414729 None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:08.171320 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7efcad0-48e2-4dc9-b634-f2308e36d267 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=86827) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:35:08.171817 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7efcad0-48e2-4dc9-b634-f2308e36d267 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:08.173445 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7efcad0-48e2-4dc9-b634-f2308e36d267 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:08.205293 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 324/633] 10.222.0.36 () {56 vars in 1137 bytes} [Wed Jul 27 10:35:08 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:35:08.206369 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7efcad0-48e2-4dc9-b634-f2308e36d267 None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:08.286977 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:08.290360 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca3b1ac-71fc-4ca7-aa61-658bd1414729 None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:08.314209 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7efcad0-48e2-4dc9-b634-f2308e36d267 None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:08.327386 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1423f7f2-0697-4d24-b28d-66321f6afe19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:08.328152 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1423f7f2-0697-4d24-b28d-66321f6afe19 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:08.328554 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1423f7f2-0697-4d24-b28d-66321f6afe19 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:08.328931 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1423f7f2-0697-4d24-b28d-66321f6afe19 None admin] PATH_INFO: `/v3/roles` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:08.329757 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1423f7f2-0697-4d24-b28d-66321f6afe19 None admin] RBAC: Adding query filter params (name=service) {{(pid=86828) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:35:08.330166 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1423f7f2-0697-4d24-b28d-66321f6afe19 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:08.331742 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1423f7f2-0697-4d24-b28d-66321f6afe19 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:08.340614 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 310/634] 10.222.0.36 () {56 vars in 1103 bytes} [Wed Jul 27 10:35:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:35:08.341539 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1423f7f2-0697-4d24-b28d-66321f6afe19 None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:08.352250 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e2ec57-6cf3-408b-9ff3-a3676a7b8cdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:08.353022 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e2ec57-6cf3-408b-9ff3-a3676a7b8cdc None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:08.353321 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e2ec57-6cf3-408b-9ff3-a3676a7b8cdc None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:08.353606 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e2ec57-6cf3-408b-9ff3-a3676a7b8cdc None admin] PATH_INFO: `/v3/roles/service` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:08.364987 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e2ec57-6cf3-408b-9ff3-a3676a7b8cdc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 10:35:08.368440 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:08.371847 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application [None req-e4e2ec57-6cf3-408b-9ff3-a3676a7b8cdc None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 27 10:35:08.371847 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:35:08.371847 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:35:08.371847 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:35:08.371847 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:35:08.371847 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:35:08.371847 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:35:08.371847 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:35:08.371847 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:35:08.371847 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:35:08.371847 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:35:08.371847 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:35:08.371847 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 10:35:08.371847 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 10:35:08.371847 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 10:35:08.371847 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 10:35:08.372780 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 10:35:08.372780 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 10:35:08.372780 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 10:35:08.372780 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 10:35:08.372780 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:35:08.372780 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:35:08.372780 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:35:08.372780 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:35:08.372780 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:35:08.372780 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:35:08.372780 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:35:08.372780 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application with Lock( Jul 27 10:35:08.372780 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:35:08.372780 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:35:08.372780 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:35:08.372780 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:35:08.373692 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:35:08.373692 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:35:08.373692 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:35:08.373692 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:35:08.373692 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 10:35:08.373692 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 10:35:08.373692 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 10:35:08.373692 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 10:35:08.373692 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 10:35:08.373692 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 10:35:08.373692 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 27 10:35:08.373692 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Jul 27 10:35:08.376123 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 325/635] 10.222.0.36 () {56 vars in 1102 bytes} [Wed Jul 27 10:35:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 10:35:08.377061 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e2ec57-6cf3-408b-9ff3-a3676a7b8cdc None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:08.447033 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1423f7f2-0697-4d24-b28d-66321f6afe19 None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:08.489936 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e2ec57-6cf3-408b-9ff3-a3676a7b8cdc None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:08.496138 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f86dd6-4315-45f4-b9f1-909931b7fd5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:08.496911 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f86dd6-4315-45f4-b9f1-909931b7fd5c None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:08.497289 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f86dd6-4315-45f4-b9f1-909931b7fd5c None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:08.497666 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f86dd6-4315-45f4-b9f1-909931b7fd5c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:08.519819 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:08.529865 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17277ed4-db43-471b-87f1-1a5fac1848e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:08.530773 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17277ed4-db43-471b-87f1-1a5fac1848e0 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:08.531206 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17277ed4-db43-471b-87f1-1a5fac1848e0 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:08.531634 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17277ed4-db43-471b-87f1-1a5fac1848e0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:08.543380 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17277ed4-db43-471b-87f1-1a5fac1848e0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:08.545065 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17277ed4-db43-471b-87f1-1a5fac1848e0 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:08.562047 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application [None req-17277ed4-db43-471b-87f1-1a5fac1848e0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 10:35:08.562047 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:35:08.562047 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:35:08.562047 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:35:08.562047 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:35:08.562047 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:35:08.562047 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:35:08.562047 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:35:08.562047 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:35:08.562047 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:35:08.562047 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:35:08.562047 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:35:08.562047 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 10:35:08.562047 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 10:35:08.562047 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 10:35:08.563199 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 10:35:08.563199 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:35:08.563199 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:35:08.563199 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:35:08.563199 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:35:08.563199 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:35:08.563199 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:35:08.563199 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:35:08.563199 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application with Lock( Jul 27 10:35:08.563199 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:35:08.563199 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:35:08.563199 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:35:08.563199 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:35:08.563199 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:35:08.563199 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:35:08.563199 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:35:08.563199 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:35:08.564117 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 10:35:08.564117 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 10:35:08.564117 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 10:35:08.564117 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Jul 27 10:35:08.566121 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 326/636] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 10:35:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 10:35:08.567113 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17277ed4-db43-471b-87f1-1a5fac1848e0 None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:08.574150 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f86dd6-4315-45f4-b9f1-909931b7fd5c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:08.575740 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f86dd6-4315-45f4-b9f1-909931b7fd5c None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:08.603180 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:08.674643 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17277ed4-db43-471b-87f1-1a5fac1848e0 None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:08.688473 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 311/637] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jul 27 10:35:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 10:35:08.689543 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f86dd6-4315-45f4-b9f1-909931b7fd5c None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:08.711544 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d838437c-db6e-49bc-b25e-7081b0ca47ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:08.712520 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d838437c-db6e-49bc-b25e-7081b0ca47ee None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:08.713010 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d838437c-db6e-49bc-b25e-7081b0ca47ee None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:08.713513 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d838437c-db6e-49bc-b25e-7081b0ca47ee None admin] PATH_INFO: `/v3/roles` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:08.714431 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d838437c-db6e-49bc-b25e-7081b0ca47ee None admin] RBAC: Adding query filter params (name=service) {{(pid=86827) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:35:08.715008 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d838437c-db6e-49bc-b25e-7081b0ca47ee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:08.716602 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d838437c-db6e-49bc-b25e-7081b0ca47ee None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:08.726192 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 327/638] 10.222.0.36 () {56 vars in 1103 bytes} [Wed Jul 27 10:35:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:35:08.729307 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d838437c-db6e-49bc-b25e-7081b0ca47ee None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:08.773470 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:08.795486 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f86dd6-4315-45f4-b9f1-909931b7fd5c None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:08.821522 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:08.834271 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7546fc-ad3f-4981-8354-210f3c9b26ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:08.835461 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7546fc-ad3f-4981-8354-210f3c9b26ce None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:08.835740 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7546fc-ad3f-4981-8354-210f3c9b26ce None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:08.836013 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7546fc-ad3f-4981-8354-210f3c9b26ce None admin] PATH_INFO: `/v3/domains` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:08.836712 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7546fc-ad3f-4981-8354-210f3c9b26ce None admin] RBAC: Adding query filter params (name=Default) {{(pid=86828) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:35:08.837020 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7546fc-ad3f-4981-8354-210f3c9b26ce None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:08.838389 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7546fc-ad3f-4981-8354-210f3c9b26ce None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:08.841053 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d838437c-db6e-49bc-b25e-7081b0ca47ee None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:08.851527 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 312/639] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 10:35:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:35:08.852510 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7546fc-ad3f-4981-8354-210f3c9b26ce None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:08.877004 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbf4e81-f856-4b4e-9bee-43200b1ae4a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:08.877581 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbf4e81-f856-4b4e-9bee-43200b1ae4a8 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:08.877865 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbf4e81-f856-4b4e-9bee-43200b1ae4a8 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:08.878136 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbf4e81-f856-4b4e-9bee-43200b1ae4a8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:08.889383 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbf4e81-f856-4b4e-9bee-43200b1ae4a8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:08.890862 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbf4e81-f856-4b4e-9bee-43200b1ae4a8 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:08.905314 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application [None req-9fbf4e81-f856-4b4e-9bee-43200b1ae4a8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 10:35:08.905314 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:35:08.905314 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:35:08.905314 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:35:08.905314 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:35:08.905314 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:35:08.905314 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:35:08.905314 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:35:08.905314 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:35:08.905314 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:35:08.905314 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:35:08.905314 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:35:08.905314 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 10:35:08.905314 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 10:35:08.905314 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 10:35:08.906208 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 10:35:08.906208 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:35:08.906208 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:35:08.906208 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:35:08.906208 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:35:08.906208 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:35:08.906208 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:35:08.906208 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:35:08.906208 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application with Lock( Jul 27 10:35:08.906208 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:35:08.906208 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:35:08.906208 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:35:08.906208 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:35:08.906208 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:35:08.906208 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:35:08.906208 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:35:08.906208 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:35:08.907120 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 10:35:08.907120 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 10:35:08.907120 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 10:35:08.907120 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Jul 27 10:35:08.907120 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 328/640] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 10:35:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 10:35:08.907903 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27773e2-0cdf-4c2c-8f2a-8fa58b21f588 None None] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:08.907903 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27773e2-0cdf-4c2c-8f2a-8fa58b21f588 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:08.908389 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27773e2-0cdf-4c2c-8f2a-8fa58b21f588 None None] PATH_INFO: `/` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:08.913705 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 329/641] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 10:35:08 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 10:35:08.913705 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27773e2-0cdf-4c2c-8f2a-8fa58b21f588 None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:08.971063 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7546fc-ad3f-4981-8354-210f3c9b26ce None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:09.009174 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e319b441-e590-48f1-b444-f36de3636595 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:09.009823 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e319b441-e590-48f1-b444-f36de3636595 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:09.010107 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e319b441-e590-48f1-b444-f36de3636595 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:09.010387 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e319b441-e590-48f1-b444-f36de3636595 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:09.020154 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:09.022570 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e319b441-e590-48f1-b444-f36de3636595 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:09.024056 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e319b441-e590-48f1-b444-f36de3636595 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:09.030752 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27773e2-0cdf-4c2c-8f2a-8fa58b21f588 None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:09.039700 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application [None req-e319b441-e590-48f1-b444-f36de3636595 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 10:35:09.039700 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:35:09.039700 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:35:09.039700 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:35:09.039700 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:35:09.039700 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:35:09.039700 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:35:09.039700 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:35:09.039700 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:35:09.039700 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:35:09.039700 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:35:09.039700 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:35:09.039700 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 10:35:09.039700 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 10:35:09.039700 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 10:35:09.040649 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 10:35:09.040649 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:35:09.040649 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:35:09.040649 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:35:09.040649 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:35:09.040649 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:35:09.040649 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:35:09.040649 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:35:09.040649 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application with Lock( Jul 27 10:35:09.040649 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:35:09.040649 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:35:09.040649 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:35:09.040649 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:35:09.040649 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:35:09.040649 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:35:09.040649 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:35:09.040649 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:35:09.041560 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 10:35:09.041560 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 10:35:09.041560 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 10:35:09.041560 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Jul 27 10:35:09.042486 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 313/642] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 10:35:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 10:35:09.043769 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c431ba3-4e0d-4fc6-bd0a-f11a501dce2a None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:09.044054 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c431ba3-4e0d-4fc6-bd0a-f11a501dce2a None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:09.044331 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c431ba3-4e0d-4fc6-bd0a-f11a501dce2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:09.062324 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-8c431ba3-4e0d-4fc6-bd0a-f11a501dce2a None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:35:09.063521 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-8c431ba3-4e0d-4fc6-bd0a-f11a501dce2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:35:09.072128 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcabb334-3da3-4160-a173-deb35e6f2e1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:09.073038 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcabb334-3da3-4160-a173-deb35e6f2e1e None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:09.073539 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcabb334-3da3-4160-a173-deb35e6f2e1e None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:09.074000 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcabb334-3da3-4160-a173-deb35e6f2e1e None admin] PATH_INFO: `/v3/users/nova` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:09.081806 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcabb334-3da3-4160-a173-deb35e6f2e1e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:09.083277 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcabb334-3da3-4160-a173-deb35e6f2e1e None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:09.094207 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:09.094812 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application [None req-fcabb334-3da3-4160-a173-deb35e6f2e1e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 27 10:35:09.094812 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:35:09.094812 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:35:09.094812 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:35:09.094812 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:35:09.094812 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:35:09.094812 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:35:09.094812 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:35:09.094812 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:35:09.094812 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:35:09.094812 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:35:09.094812 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:35:09.094812 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 10:35:09.094812 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 10:35:09.094812 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 10:35:09.094812 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 10:35:09.095702 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:35:09.095702 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:35:09.095702 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 10:35:09.095702 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 10:35:09.095702 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 10:35:09.095702 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 10:35:09.095702 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:35:09.095702 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:35:09.095702 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:35:09.095702 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:35:09.095702 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:35:09.095702 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application with Lock( Jul 27 10:35:09.095702 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:35:09.095702 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:35:09.095702 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:35:09.095702 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:35:09.095702 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:35:09.095702 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:35:09.096740 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:35:09.096740 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:35:09.096740 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 10:35:09.096740 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 10:35:09.096740 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 10:35:09.096740 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 10:35:09.096740 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 10:35:09.096740 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 10:35:09.096740 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 27 10:35:09.096740 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Jul 27 10:35:09.096740 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 330/643] 10.222.0.36 () {56 vars in 1093 bytes} [Wed Jul 27 10:35:08 2022] GET /identity/v3/users/nova => generated 82 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 10:35:09.096740 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcabb334-3da3-4160-a173-deb35e6f2e1e None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:09.139434 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 314/644] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 10:35:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 10:35:09.139978 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c431ba3-4e0d-4fc6-bd0a-f11a501dce2a None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:09.217663 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcabb334-3da3-4160-a173-deb35e6f2e1e None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:09.241665 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c431ba3-4e0d-4fc6-bd0a-f11a501dce2a None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:09.253216 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a213a1b2-914b-4db2-bc89-daf7d8146ea4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:09.253830 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a213a1b2-914b-4db2-bc89-daf7d8146ea4 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:09.254117 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a213a1b2-914b-4db2-bc89-daf7d8146ea4 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:09.254395 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a213a1b2-914b-4db2-bc89-daf7d8146ea4 None admin] PATH_INFO: `/v3/domains` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:09.255125 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a213a1b2-914b-4db2-bc89-daf7d8146ea4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86827) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:35:09.255460 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a213a1b2-914b-4db2-bc89-daf7d8146ea4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:09.256836 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a213a1b2-914b-4db2-bc89-daf7d8146ea4 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:09.274162 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 331/645] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 10:35:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:35:09.274162 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a213a1b2-914b-4db2-bc89-daf7d8146ea4 None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:09.277557 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7720089-37a1-4ffb-8942-be64e5d295ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:09.278193 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7720089-37a1-4ffb-8942-be64e5d295ac None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:09.278483 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7720089-37a1-4ffb-8942-be64e5d295ac None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:09.278809 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7720089-37a1-4ffb-8942-be64e5d295ac None admin] PATH_INFO: `/v3/domains` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:09.279514 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7720089-37a1-4ffb-8942-be64e5d295ac None admin] RBAC: Adding query filter params (name=Default) {{(pid=86828) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:35:09.279827 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7720089-37a1-4ffb-8942-be64e5d295ac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:09.281197 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7720089-37a1-4ffb-8942-be64e5d295ac None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:09.290460 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:09.294961 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 315/646] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 10:35:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:35:09.296002 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7720089-37a1-4ffb-8942-be64e5d295ac None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:09.323779 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:09.387686 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a213a1b2-914b-4db2-bc89-daf7d8146ea4 None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:09.407303 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7720089-37a1-4ffb-8942-be64e5d295ac None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:09.422448 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05852af-50e3-4f13-9c7d-ebb7878834ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:09.423117 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05852af-50e3-4f13-9c7d-ebb7878834ea None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:09.423401 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05852af-50e3-4f13-9c7d-ebb7878834ea None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:09.423676 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05852af-50e3-4f13-9c7d-ebb7878834ea None admin] PATH_INFO: `/v3/users` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:09.424474 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05852af-50e3-4f13-9c7d-ebb7878834ea None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86827) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:35:09.424782 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05852af-50e3-4f13-9c7d-ebb7878834ea None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:09.426222 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05852af-50e3-4f13-9c7d-ebb7878834ea None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:09.444230 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 332/647] 10.222.0.36 () {56 vars in 1133 bytes} [Wed Jul 27 10:35:09 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:35:09.445047 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05852af-50e3-4f13-9c7d-ebb7878834ea None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:09.445618 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b25c0dc-6a7e-46ad-afa2-94254efac816 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:09.446447 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b25c0dc-6a7e-46ad-afa2-94254efac816 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:09.446906 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b25c0dc-6a7e-46ad-afa2-94254efac816 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:09.447337 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b25c0dc-6a7e-46ad-afa2-94254efac816 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:09.458971 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b25c0dc-6a7e-46ad-afa2-94254efac816 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 10:35:09.464529 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application [None req-9b25c0dc-6a7e-46ad-afa2-94254efac816 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 27 10:35:09.464529 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:35:09.464529 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:35:09.464529 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:35:09.464529 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:35:09.464529 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:35:09.464529 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:35:09.464529 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:35:09.464529 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:35:09.464529 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:35:09.464529 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:35:09.464529 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:35:09.464529 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 10:35:09.464529 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 10:35:09.464529 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 10:35:09.464529 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 10:35:09.465428 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 10:35:09.465428 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 10:35:09.465428 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 10:35:09.465428 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 10:35:09.465428 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:35:09.465428 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:35:09.465428 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:35:09.465428 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:35:09.465428 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:35:09.465428 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:35:09.465428 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:35:09.465428 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application with Lock( Jul 27 10:35:09.465428 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:35:09.465428 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:35:09.465428 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:35:09.465428 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:35:09.466238 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:35:09.466238 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:35:09.466238 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:35:09.466238 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:35:09.466238 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 10:35:09.466238 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 10:35:09.466238 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 10:35:09.466238 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 10:35:09.466238 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 10:35:09.466238 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 10:35:09.466238 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 27 10:35:09.466238 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Jul 27 10:35:09.468635 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 316/648] 10.222.0.36 () {56 vars in 1102 bytes} [Wed Jul 27 10:35:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 10:35:09.469650 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b25c0dc-6a7e-46ad-afa2-94254efac816 None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:09.522365 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:09.554301 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05852af-50e3-4f13-9c7d-ebb7878834ea None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:09.563307 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:09.580544 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b25c0dc-6a7e-46ad-afa2-94254efac816 None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:09.592573 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57c9d35-d6e4-4813-ad1a-464aeef23e0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:09.593207 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57c9d35-d6e4-4813-ad1a-464aeef23e0a None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:09.593493 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57c9d35-d6e4-4813-ad1a-464aeef23e0a None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:09.593823 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57c9d35-d6e4-4813-ad1a-464aeef23e0a None admin] PATH_INFO: `/v3/projects/service` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:09.601413 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57c9d35-d6e4-4813-ad1a-464aeef23e0a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:09.602892 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57c9d35-d6e4-4813-ad1a-464aeef23e0a None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:09.614164 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application [None req-b57c9d35-d6e4-4813-ad1a-464aeef23e0a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 10:35:09.614164 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:35:09.614164 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:35:09.614164 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:35:09.614164 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:35:09.614164 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:35:09.614164 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:35:09.614164 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:35:09.614164 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:35:09.614164 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:35:09.614164 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:35:09.614164 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:35:09.614164 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 10:35:09.614164 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 10:35:09.614164 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 10:35:09.615237 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 10:35:09.615237 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:35:09.615237 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:35:09.615237 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:35:09.615237 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:35:09.615237 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:35:09.615237 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:35:09.615237 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:35:09.615237 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application with Lock( Jul 27 10:35:09.615237 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:35:09.615237 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:35:09.615237 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:35:09.615237 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:35:09.615237 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:35:09.615237 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:35:09.615237 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:35:09.615237 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:35:09.616188 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 10:35:09.616188 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 10:35:09.616188 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 10:35:09.616188 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 10:35:09.616188 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 10:35:09.616188 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 10:35:09.616188 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 10:35:09.616188 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Jul 27 10:35:09.616188 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 333/649] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Jul 27 10:35:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 10:35:09.616679 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57c9d35-d6e4-4813-ad1a-464aeef23e0a None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:09.622536 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d86269f-7173-44c2-9eb0-3f014f2bcb89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:09.623217 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d86269f-7173-44c2-9eb0-3f014f2bcb89 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:09.623497 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d86269f-7173-44c2-9eb0-3f014f2bcb89 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:09.623765 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d86269f-7173-44c2-9eb0-3f014f2bcb89 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:09.631707 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d86269f-7173-44c2-9eb0-3f014f2bcb89 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:09.633126 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d86269f-7173-44c2-9eb0-3f014f2bcb89 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:09.645558 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application [None req-2d86269f-7173-44c2-9eb0-3f014f2bcb89 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 27 10:35:09.645558 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:35:09.645558 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:35:09.645558 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:35:09.645558 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:35:09.645558 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:35:09.645558 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:35:09.645558 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:35:09.645558 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:35:09.645558 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:35:09.645558 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:35:09.645558 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:35:09.645558 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 10:35:09.645558 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 10:35:09.645558 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 10:35:09.645558 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 10:35:09.646517 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:35:09.646517 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:35:09.646517 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 10:35:09.646517 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 10:35:09.646517 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 10:35:09.646517 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 10:35:09.646517 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:35:09.646517 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:35:09.646517 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:35:09.646517 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:35:09.646517 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:35:09.646517 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application with Lock( Jul 27 10:35:09.646517 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:35:09.646517 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:35:09.646517 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:35:09.646517 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:35:09.646517 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:35:09.646517 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:35:09.648016 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:35:09.648016 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:35:09.648016 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 10:35:09.648016 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 10:35:09.648016 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 10:35:09.648016 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 10:35:09.648016 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 10:35:09.648016 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 10:35:09.648016 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 27 10:35:09.648016 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Jul 27 10:35:09.648016 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 317/650] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jul 27 10:35:09 2022] GET /identity/v3/users/cinder => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 10:35:09.648016 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d86269f-7173-44c2-9eb0-3f014f2bcb89 None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:09.735869 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57c9d35-d6e4-4813-ad1a-464aeef23e0a None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:09.747389 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d86269f-7173-44c2-9eb0-3f014f2bcb89 None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:09.765858 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:09.772213 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb0581b-6dfc-498c-b636-a7a8ba90ad09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:09.772842 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb0581b-6dfc-498c-b636-a7a8ba90ad09 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:09.773120 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb0581b-6dfc-498c-b636-a7a8ba90ad09 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:09.773402 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb0581b-6dfc-498c-b636-a7a8ba90ad09 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:09.783470 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc0cae8-008c-43f6-b444-002538ec8d1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:09.784097 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc0cae8-008c-43f6-b444-002538ec8d1c None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:09.784381 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc0cae8-008c-43f6-b444-002538ec8d1c None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:09.784665 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc0cae8-008c-43f6-b444-002538ec8d1c None admin] PATH_INFO: `/v3/roles` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:09.785387 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc0cae8-008c-43f6-b444-002538ec8d1c None admin] RBAC: Adding query filter params (name=service) {{(pid=86828) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:35:09.785702 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc0cae8-008c-43f6-b444-002538ec8d1c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:09.787110 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc0cae8-008c-43f6-b444-002538ec8d1c None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:09.800539 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 318/651] 10.222.0.36 () {56 vars in 1103 bytes} [Wed Jul 27 10:35:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:35:09.800539 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc0cae8-008c-43f6-b444-002538ec8d1c None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:09.810256 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:09.854371 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb0581b-6dfc-498c-b636-a7a8ba90ad09 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:09.855879 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb0581b-6dfc-498c-b636-a7a8ba90ad09 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:09.906654 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc0cae8-008c-43f6-b444-002538ec8d1c None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:09.939846 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d850a131-1f07-474f-97a1-4986283c1eeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:09.940472 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d850a131-1f07-474f-97a1-4986283c1eeb None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:09.940758 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d850a131-1f07-474f-97a1-4986283c1eeb None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:09.941027 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d850a131-1f07-474f-97a1-4986283c1eeb None admin] PATH_INFO: `/v3/projects` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:09.942019 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d850a131-1f07-474f-97a1-4986283c1eeb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86828) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:35:09.942113 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d850a131-1f07-474f-97a1-4986283c1eeb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:09.943529 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d850a131-1f07-474f-97a1-4986283c1eeb None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:09.956394 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 319/652] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Jul 27 10:35:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:35:09.957201 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d850a131-1f07-474f-97a1-4986283c1eeb None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:09.966096 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 334/653] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jul 27 10:35:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 10:35:09.967112 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb0581b-6dfc-498c-b636-a7a8ba90ad09 None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:09.999088 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:10.023947 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:10.056190 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d850a131-1f07-474f-97a1-4986283c1eeb None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:10.067814 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb0581b-6dfc-498c-b636-a7a8ba90ad09 None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:10.089988 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca87fc0b-db81-4ab0-b311-38642682ff24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:10.090622 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca87fc0b-db81-4ab0-b311-38642682ff24 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:10.090929 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca87fc0b-db81-4ab0-b311-38642682ff24 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:10.091207 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca87fc0b-db81-4ab0-b311-38642682ff24 None admin] PATH_INFO: `/v3/users` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:10.092003 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca87fc0b-db81-4ab0-b311-38642682ff24 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=86828) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:35:10.092310 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca87fc0b-db81-4ab0-b311-38642682ff24 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:10.093731 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca87fc0b-db81-4ab0-b311-38642682ff24 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:10.102172 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b217fbf3-d729-4a57-9a20-8950e60adbdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:10.102830 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b217fbf3-d729-4a57-9a20-8950e60adbdf None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:10.103109 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b217fbf3-d729-4a57-9a20-8950e60adbdf None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:10.103381 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b217fbf3-d729-4a57-9a20-8950e60adbdf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:10.110877 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 320/654] 10.222.0.36 () {56 vars in 1137 bytes} [Wed Jul 27 10:35:09 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:35:10.112064 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65ebbc6-634c-46b0-8fb4-407c395cf247 None None] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:10.112358 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65ebbc6-634c-46b0-8fb4-407c395cf247 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:10.112635 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65ebbc6-634c-46b0-8fb4-407c395cf247 None None] PATH_INFO: `/` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:10.113264 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b217fbf3-d729-4a57-9a20-8950e60adbdf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:10.114042 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 321/655] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 10:35:10 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 10:35:10.114542 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65ebbc6-634c-46b0-8fb4-407c395cf247 None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:10.118543 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b217fbf3-d729-4a57-9a20-8950e60adbdf None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:10.134634 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application [None req-b217fbf3-d729-4a57-9a20-8950e60adbdf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 10:35:10.134634 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:35:10.134634 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:35:10.134634 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:35:10.134634 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:35:10.134634 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:35:10.134634 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:35:10.134634 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:35:10.134634 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:35:10.134634 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:35:10.134634 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:35:10.134634 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:35:10.134634 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 10:35:10.134634 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 10:35:10.134634 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 10:35:10.135507 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 10:35:10.135507 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:35:10.135507 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:35:10.135507 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:35:10.135507 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:35:10.135507 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:35:10.135507 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:35:10.135507 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:35:10.135507 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application with Lock( Jul 27 10:35:10.135507 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:35:10.135507 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:35:10.135507 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:35:10.135507 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:35:10.135507 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:35:10.135507 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:35:10.135507 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:35:10.135507 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:35:10.136644 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 10:35:10.136644 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 10:35:10.136644 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 10:35:10.136644 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Jul 27 10:35:10.136644 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 335/656] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 10:35:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 10:35:10.137156 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b217fbf3-d729-4a57-9a20-8950e60adbdf None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:10.215824 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65ebbc6-634c-46b0-8fb4-407c395cf247 None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:10.216663 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:10.237700 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b217fbf3-d729-4a57-9a20-8950e60adbdf None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:10.243443 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:10.252469 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fff9db-f24a-4ce3-8bb9-82229944f6bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:10.253024 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fff9db-f24a-4ce3-8bb9-82229944f6bb None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:10.253303 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fff9db-f24a-4ce3-8bb9-82229944f6bb None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:10.253574 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fff9db-f24a-4ce3-8bb9-82229944f6bb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:10.254358 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fff9db-f24a-4ce3-8bb9-82229944f6bb None admin] RBAC: Adding query filter params (role.id=9e0038c6bbd34556af3f84f9e2a7c2c5, scope.project.id=5f296d8e3b8f4001909addc49d8dac01, user.id=bc03c5d232654878aa1ddce15f49fdaa) {{(pid=86828) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:35:10.254696 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fff9db-f24a-4ce3-8bb9-82229944f6bb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:10.256114 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fff9db-f24a-4ce3-8bb9-82229944f6bb None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:10.262022 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 322/657] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Jul 27 10:35:10 2022] GET /identity/v3/role_assignments?user.id=bc03c5d232654878aa1ddce15f49fdaa&scope.project.id=5f296d8e3b8f4001909addc49d8dac01&role.id=9e0038c6bbd34556af3f84f9e2a7c2c5 => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:35:10.262807 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fff9db-f24a-4ce3-8bb9-82229944f6bb None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:10.271910 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4f7ac2-73a7-49d3-be3e-0b4ef3434a8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:10.272630 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4f7ac2-73a7-49d3-be3e-0b4ef3434a8c None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:10.272915 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4f7ac2-73a7-49d3-be3e-0b4ef3434a8c None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:10.273193 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4f7ac2-73a7-49d3-be3e-0b4ef3434a8c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:10.285151 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4f7ac2-73a7-49d3-be3e-0b4ef3434a8c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:10.286659 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4f7ac2-73a7-49d3-be3e-0b4ef3434a8c None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:10.305462 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application [None req-5e4f7ac2-73a7-49d3-be3e-0b4ef3434a8c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 10:35:10.305462 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:35:10.305462 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:35:10.305462 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:35:10.305462 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:35:10.305462 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:35:10.305462 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:35:10.305462 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:35:10.305462 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:35:10.305462 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:35:10.305462 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:35:10.305462 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:35:10.305462 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 10:35:10.305462 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 10:35:10.305462 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 10:35:10.312005 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 10:35:10.312005 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:35:10.312005 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:35:10.312005 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:35:10.312005 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:35:10.312005 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:35:10.312005 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:35:10.312005 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:35:10.312005 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application with Lock( Jul 27 10:35:10.312005 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:35:10.312005 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:35:10.312005 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:35:10.312005 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:35:10.312005 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:35:10.312005 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:35:10.312005 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:35:10.312005 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:35:10.312875 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 10:35:10.312875 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 10:35:10.312875 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 10:35:10.312875 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Jul 27 10:35:10.312875 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 336/658] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 10:35:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 10:35:10.312875 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06933e2c-0190-4561-918d-65b13d20cc8f None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:10.312875 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06933e2c-0190-4561-918d-65b13d20cc8f None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:10.312875 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06933e2c-0190-4561-918d-65b13d20cc8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:10.328207 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-06933e2c-0190-4561-918d-65b13d20cc8f None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:35:10.329442 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-06933e2c-0190-4561-918d-65b13d20cc8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:35:10.385827 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fff9db-f24a-4ce3-8bb9-82229944f6bb None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:10.410685 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 337/659] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 10:35:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 10:35:10.410685 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06933e2c-0190-4561-918d-65b13d20cc8f None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:10.427557 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4c27ce-b894-40a4-abab-9c225ee76ce4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:10.428404 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4c27ce-b894-40a4-abab-9c225ee76ce4 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:10.428748 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4c27ce-b894-40a4-abab-9c225ee76ce4 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:10.429115 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4c27ce-b894-40a4-abab-9c225ee76ce4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:10.477661 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:10.498391 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:10.510399 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4c27ce-b894-40a4-abab-9c225ee76ce4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:10.511835 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4c27ce-b894-40a4-abab-9c225ee76ce4 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:10.526870 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06933e2c-0190-4561-918d-65b13d20cc8f None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:10.562614 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f61216c-1a01-43b0-b534-291e1a90b5ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:10.563253 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f61216c-1a01-43b0-b534-291e1a90b5ac None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:10.563529 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f61216c-1a01-43b0-b534-291e1a90b5ac None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:10.563807 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f61216c-1a01-43b0-b534-291e1a90b5ac None admin] PATH_INFO: `/v3/domains` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:10.564490 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f61216c-1a01-43b0-b534-291e1a90b5ac None admin] RBAC: Adding query filter params (name=Default) {{(pid=86827) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:35:10.564804 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f61216c-1a01-43b0-b534-291e1a90b5ac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:10.566163 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f61216c-1a01-43b0-b534-291e1a90b5ac None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:10.577900 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 338/660] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 10:35:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:35:10.579760 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f61216c-1a01-43b0-b534-291e1a90b5ac None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:10.618054 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 323/661] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jul 27 10:35:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 10:35:10.619009 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4c27ce-b894-40a4-abab-9c225ee76ce4 None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:10.695458 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f61216c-1a01-43b0-b534-291e1a90b5ac None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:10.718664 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:10.719158 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:10.729700 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4c27ce-b894-40a4-abab-9c225ee76ce4 None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:10.731244 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7351cb04-31f4-4f33-968f-dcb5691d74da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:10.731921 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7351cb04-31f4-4f33-968f-dcb5691d74da None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:10.732234 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7351cb04-31f4-4f33-968f-dcb5691d74da None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:10.732516 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7351cb04-31f4-4f33-968f-dcb5691d74da None admin] PATH_INFO: `/v3/domains` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:10.733223 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7351cb04-31f4-4f33-968f-dcb5691d74da None admin] RBAC: Adding query filter params (name=Default) {{(pid=86827) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:35:10.733555 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7351cb04-31f4-4f33-968f-dcb5691d74da None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:10.735055 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7351cb04-31f4-4f33-968f-dcb5691d74da None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:10.752669 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 339/662] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 10:35:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:35:10.752669 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7351cb04-31f4-4f33-968f-dcb5691d74da None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:10.768459 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e11312-525b-4fcf-a65d-67d759ed2c8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:10.769033 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e11312-525b-4fcf-a65d-67d759ed2c8a None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:10.769309 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e11312-525b-4fcf-a65d-67d759ed2c8a None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:10.769573 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e11312-525b-4fcf-a65d-67d759ed2c8a None admin] PATH_INFO: `/v3/roles/service` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:10.780705 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e11312-525b-4fcf-a65d-67d759ed2c8a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 10:35:10.785700 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application [None req-d4e11312-525b-4fcf-a65d-67d759ed2c8a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 27 10:35:10.785700 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:35:10.785700 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:35:10.785700 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:35:10.785700 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:35:10.785700 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:35:10.785700 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:35:10.785700 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:35:10.785700 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:35:10.785700 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:35:10.785700 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:35:10.785700 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:35:10.785700 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 10:35:10.785700 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 10:35:10.785700 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 10:35:10.785700 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 10:35:10.786673 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 10:35:10.786673 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 10:35:10.786673 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 10:35:10.786673 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 10:35:10.786673 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:35:10.786673 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:35:10.786673 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:35:10.786673 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:35:10.786673 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:35:10.786673 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:35:10.786673 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:35:10.786673 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application with Lock( Jul 27 10:35:10.786673 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:35:10.786673 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:35:10.786673 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:35:10.786673 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:35:10.787547 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:35:10.787547 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:35:10.787547 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:35:10.787547 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:35:10.787547 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 10:35:10.787547 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 10:35:10.787547 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 10:35:10.787547 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 10:35:10.787547 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 10:35:10.787547 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 10:35:10.787547 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 27 10:35:10.787547 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Jul 27 10:35:10.787547 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 324/663] 10.222.0.36 () {56 vars in 1102 bytes} [Wed Jul 27 10:35:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 10:35:10.788311 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e11312-525b-4fcf-a65d-67d759ed2c8a None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:10.865467 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7351cb04-31f4-4f33-968f-dcb5691d74da None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:10.899255 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e11312-525b-4fcf-a65d-67d759ed2c8a None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:10.901678 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce89931e-bf4a-42db-9077-2418e55111c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:10.902331 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce89931e-bf4a-42db-9077-2418e55111c3 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:10.902633 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce89931e-bf4a-42db-9077-2418e55111c3 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:10.902918 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce89931e-bf4a-42db-9077-2418e55111c3 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:10.910071 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce89931e-bf4a-42db-9077-2418e55111c3 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:10.911544 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce89931e-bf4a-42db-9077-2418e55111c3 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:10.923316 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application [None req-ce89931e-bf4a-42db-9077-2418e55111c3 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 27 10:35:10.923316 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:35:10.923316 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:35:10.923316 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:35:10.923316 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:35:10.923316 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:35:10.923316 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:35:10.923316 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:35:10.923316 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:35:10.923316 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:35:10.923316 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:35:10.923316 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:35:10.923316 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 10:35:10.923316 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 10:35:10.923316 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 10:35:10.923316 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 10:35:10.924349 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:35:10.924349 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:35:10.924349 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 10:35:10.924349 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 10:35:10.924349 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 10:35:10.924349 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 10:35:10.924349 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:35:10.924349 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:35:10.924349 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:35:10.924349 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:35:10.924349 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:35:10.924349 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application with Lock( Jul 27 10:35:10.924349 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:35:10.924349 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:35:10.924349 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:35:10.924349 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:35:10.924349 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:35:10.924349 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:35:10.925274 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:35:10.925274 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:35:10.925274 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 10:35:10.925274 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 10:35:10.925274 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 10:35:10.925274 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 10:35:10.925274 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 10:35:10.925274 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 10:35:10.925274 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 27 10:35:10.925274 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Jul 27 10:35:10.925274 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 340/664] 10.222.0.36 () {56 vars in 1102 bytes} [Wed Jul 27 10:35:10 2022] GET /identity/v3/users/neutron => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 10:35:10.925274 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce89931e-bf4a-42db-9077-2418e55111c3 None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:10.938842 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472285ec-8e82-4843-8f04-edabad3f086e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:10.939494 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472285ec-8e82-4843-8f04-edabad3f086e None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:10.939780 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472285ec-8e82-4843-8f04-edabad3f086e None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:10.940055 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472285ec-8e82-4843-8f04-edabad3f086e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:10.953842 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472285ec-8e82-4843-8f04-edabad3f086e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:10.955421 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472285ec-8e82-4843-8f04-edabad3f086e None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:10.972105 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application [None req-472285ec-8e82-4843-8f04-edabad3f086e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 10:35:10.972105 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:35:10.972105 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:35:10.972105 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:35:10.972105 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:35:10.972105 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:35:10.972105 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:35:10.972105 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:35:10.972105 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:35:10.972105 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:35:10.972105 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:35:10.972105 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:35:10.972105 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 10:35:10.972105 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 10:35:10.972105 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 10:35:10.973002 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 10:35:10.973002 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:35:10.973002 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:35:10.973002 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:35:10.973002 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:35:10.973002 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:35:10.973002 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:35:10.973002 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:35:10.973002 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application with Lock( Jul 27 10:35:10.973002 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:35:10.973002 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:35:10.973002 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:35:10.973002 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:35:10.973002 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:35:10.973002 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:35:10.973002 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:35:10.973002 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:35:10.974049 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 10:35:10.974049 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 10:35:10.974049 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 10:35:10.974049 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Jul 27 10:35:10.974049 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 325/665] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 10:35:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 10:35:10.974906 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472285ec-8e82-4843-8f04-edabad3f086e None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:10.977853 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:10.980724 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:11.048727 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce89931e-bf4a-42db-9077-2418e55111c3 None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:11.083084 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472285ec-8e82-4843-8f04-edabad3f086e None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:11.084655 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379bd615-0ebc-4444-b2ec-2d393f11112f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:11.086528 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379bd615-0ebc-4444-b2ec-2d393f11112f None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:11.086528 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379bd615-0ebc-4444-b2ec-2d393f11112f None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:11.086528 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379bd615-0ebc-4444-b2ec-2d393f11112f None admin] PATH_INFO: `/v3/projects/service` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:11.093571 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379bd615-0ebc-4444-b2ec-2d393f11112f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:11.094869 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379bd615-0ebc-4444-b2ec-2d393f11112f None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:11.106374 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application [None req-379bd615-0ebc-4444-b2ec-2d393f11112f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 10:35:11.106374 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:35:11.106374 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:35:11.106374 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:35:11.106374 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:35:11.106374 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:35:11.106374 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:35:11.106374 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:35:11.106374 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:35:11.106374 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:35:11.106374 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:35:11.106374 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:35:11.106374 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 10:35:11.106374 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 10:35:11.106374 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 10:35:11.107340 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 10:35:11.107340 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:35:11.107340 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:35:11.107340 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:35:11.107340 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:35:11.107340 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:35:11.107340 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:35:11.107340 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:35:11.107340 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application with Lock( Jul 27 10:35:11.107340 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:35:11.107340 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:35:11.107340 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:35:11.107340 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:35:11.107340 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:35:11.107340 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:35:11.107340 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:35:11.107340 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:35:11.113197 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 10:35:11.113197 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 10:35:11.113197 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 10:35:11.113197 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 10:35:11.113197 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 10:35:11.113197 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 10:35:11.113197 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 10:35:11.113197 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Jul 27 10:35:11.113197 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 341/666] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Jul 27 10:35:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 10:35:11.113197 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379bd615-0ebc-4444-b2ec-2d393f11112f None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:11.126964 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfe8e0b-88b8-4273-8b52-a070ebda2ea5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:11.127591 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfe8e0b-88b8-4273-8b52-a070ebda2ea5 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:11.127878 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfe8e0b-88b8-4273-8b52-a070ebda2ea5 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:11.128153 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfe8e0b-88b8-4273-8b52-a070ebda2ea5 None admin] PATH_INFO: `/v3/roles` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:11.128873 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfe8e0b-88b8-4273-8b52-a070ebda2ea5 None admin] RBAC: Adding query filter params (name=service) {{(pid=86828) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:35:11.129182 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfe8e0b-88b8-4273-8b52-a070ebda2ea5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:11.130561 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfe8e0b-88b8-4273-8b52-a070ebda2ea5 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:11.139682 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 326/667] 10.222.0.36 () {56 vars in 1103 bytes} [Wed Jul 27 10:35:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:35:11.144214 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfe8e0b-88b8-4273-8b52-a070ebda2ea5 None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:11.214659 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:11.217645 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379bd615-0ebc-4444-b2ec-2d393f11112f None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:11.221035 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:11.256176 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfe8e0b-88b8-4273-8b52-a070ebda2ea5 None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:11.256631 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b75418-8707-4257-9409-729a1da0ee47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:11.257785 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b75418-8707-4257-9409-729a1da0ee47 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:11.258061 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b75418-8707-4257-9409-729a1da0ee47 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:11.258350 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b75418-8707-4257-9409-729a1da0ee47 None admin] PATH_INFO: `/v3/users` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:11.259433 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b75418-8707-4257-9409-729a1da0ee47 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=86827) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:35:11.259743 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b75418-8707-4257-9409-729a1da0ee47 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:11.261168 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b75418-8707-4257-9409-729a1da0ee47 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:11.278067 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 342/668] 10.222.0.36 () {56 vars in 1139 bytes} [Wed Jul 27 10:35:11 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:35:11.278986 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b75418-8707-4257-9409-729a1da0ee47 None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:11.298526 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c22cda-2d68-4c9a-ba88-448be5f217a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:11.299187 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c22cda-2d68-4c9a-ba88-448be5f217a3 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:11.299475 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c22cda-2d68-4c9a-ba88-448be5f217a3 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:11.299753 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c22cda-2d68-4c9a-ba88-448be5f217a3 None admin] PATH_INFO: `/v3/domains` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:11.300448 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c22cda-2d68-4c9a-ba88-448be5f217a3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86828) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:35:11.300757 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c22cda-2d68-4c9a-ba88-448be5f217a3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:11.302130 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c22cda-2d68-4c9a-ba88-448be5f217a3 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:11.315565 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 327/669] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 10:35:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:35:11.320603 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c22cda-2d68-4c9a-ba88-448be5f217a3 None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:11.384721 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b75418-8707-4257-9409-729a1da0ee47 None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:11.421606 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633bfbeb-1bf0-4fb8-955b-025aa20fb297 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:11.422334 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633bfbeb-1bf0-4fb8-955b-025aa20fb297 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:11.422517 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633bfbeb-1bf0-4fb8-955b-025aa20fb297 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:11.422823 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633bfbeb-1bf0-4fb8-955b-025aa20fb297 None admin] PATH_INFO: `/v3/projects` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:11.423598 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633bfbeb-1bf0-4fb8-955b-025aa20fb297 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86827) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:35:11.423924 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633bfbeb-1bf0-4fb8-955b-025aa20fb297 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:11.425372 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633bfbeb-1bf0-4fb8-955b-025aa20fb297 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:11.430464 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c22cda-2d68-4c9a-ba88-448be5f217a3 None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:11.438641 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 343/670] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Jul 27 10:35:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:35:11.439497 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633bfbeb-1bf0-4fb8-955b-025aa20fb297 None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:11.461143 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:11.469606 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:11.477283 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f924206b-9770-4a9d-9020-ac6c0ab654c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:11.477935 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f924206b-9770-4a9d-9020-ac6c0ab654c2 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:11.478218 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f924206b-9770-4a9d-9020-ac6c0ab654c2 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:11.478498 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f924206b-9770-4a9d-9020-ac6c0ab654c2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:11.489799 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f924206b-9770-4a9d-9020-ac6c0ab654c2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:11.492093 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f924206b-9770-4a9d-9020-ac6c0ab654c2 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:11.508656 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application [None req-f924206b-9770-4a9d-9020-ac6c0ab654c2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 10:35:11.508656 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:35:11.508656 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:35:11.508656 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:35:11.508656 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:35:11.508656 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:35:11.508656 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:35:11.508656 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:35:11.508656 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:35:11.508656 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:35:11.508656 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:35:11.508656 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:35:11.508656 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 10:35:11.508656 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 10:35:11.508656 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 10:35:11.509907 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 10:35:11.509907 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:35:11.509907 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:35:11.509907 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:35:11.509907 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:35:11.509907 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:35:11.509907 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:35:11.509907 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:35:11.509907 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application with Lock( Jul 27 10:35:11.509907 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:35:11.509907 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:35:11.509907 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:35:11.509907 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:35:11.509907 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:35:11.509907 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:35:11.509907 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:35:11.509907 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:35:11.510838 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 10:35:11.510838 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 10:35:11.510838 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 10:35:11.510838 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Jul 27 10:35:11.510838 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 328/671] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 10:35:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 10:35:11.511151 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f924206b-9770-4a9d-9020-ac6c0ab654c2 None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:11.556091 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633bfbeb-1bf0-4fb8-955b-025aa20fb297 None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:11.592911 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540036ee-0208-49c9-8b49-083ec6548264 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:11.593527 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540036ee-0208-49c9-8b49-083ec6548264 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:11.593812 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540036ee-0208-49c9-8b49-083ec6548264 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:11.594207 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540036ee-0208-49c9-8b49-083ec6548264 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:11.617430 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f924206b-9770-4a9d-9020-ac6c0ab654c2 None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:11.650435 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9630093b-cdd2-4b66-b989-cb66276bb9ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:11.651108 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9630093b-cdd2-4b66-b989-cb66276bb9ef None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:11.651396 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9630093b-cdd2-4b66-b989-cb66276bb9ef None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:11.651677 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9630093b-cdd2-4b66-b989-cb66276bb9ef None admin] PATH_INFO: `/v3/projects/service` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:11.658761 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9630093b-cdd2-4b66-b989-cb66276bb9ef None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:11.660359 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9630093b-cdd2-4b66-b989-cb66276bb9ef None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:11.668151 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540036ee-0208-49c9-8b49-083ec6548264 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:11.669578 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540036ee-0208-49c9-8b49-083ec6548264 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:11.672862 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application [None req-9630093b-cdd2-4b66-b989-cb66276bb9ef None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 10:35:11.672862 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:35:11.672862 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:35:11.672862 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:35:11.672862 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:35:11.672862 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:35:11.672862 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:35:11.672862 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:35:11.672862 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:35:11.672862 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:35:11.672862 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:35:11.672862 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:35:11.672862 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 10:35:11.672862 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 10:35:11.672862 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 10:35:11.673972 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 10:35:11.673972 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:35:11.673972 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:35:11.673972 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:35:11.673972 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:35:11.673972 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:35:11.673972 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:35:11.673972 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:35:11.673972 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application with Lock( Jul 27 10:35:11.673972 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:35:11.673972 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:35:11.673972 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:35:11.673972 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:35:11.673972 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:35:11.673972 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:35:11.673972 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:35:11.673972 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:35:11.675119 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 10:35:11.675119 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 10:35:11.675119 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 10:35:11.675119 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 10:35:11.675119 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 10:35:11.675119 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 10:35:11.675119 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 10:35:11.675119 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Jul 27 10:35:11.677146 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 329/672] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Jul 27 10:35:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 10:35:11.677932 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9630093b-cdd2-4b66-b989-cb66276bb9ef None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:11.698113 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:11.718439 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:11.780817 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 344/673] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jul 27 10:35:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 10:35:11.781691 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540036ee-0208-49c9-8b49-083ec6548264 None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:11.795499 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9630093b-cdd2-4b66-b989-cb66276bb9ef None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:11.832353 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8ea07c-2684-4ac7-9527-594fa1a68ac7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:11.833265 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8ea07c-2684-4ac7-9527-594fa1a68ac7 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:11.833696 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8ea07c-2684-4ac7-9527-594fa1a68ac7 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:11.834101 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8ea07c-2684-4ac7-9527-594fa1a68ac7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:11.835649 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8ea07c-2684-4ac7-9527-594fa1a68ac7 None admin] RBAC: Adding query filter params (role.id=9e0038c6bbd34556af3f84f9e2a7c2c5, scope.project.id=5f296d8e3b8f4001909addc49d8dac01, user.id=e059d8b6c28c4fcba925d05d7c98e4ac) {{(pid=86828) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:35:11.835974 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8ea07c-2684-4ac7-9527-594fa1a68ac7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:11.837707 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8ea07c-2684-4ac7-9527-594fa1a68ac7 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:11.844385 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 330/674] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Jul 27 10:35:11 2022] GET /identity/v3/role_assignments?user.id=e059d8b6c28c4fcba925d05d7c98e4ac&scope.project.id=5f296d8e3b8f4001909addc49d8dac01&role.id=9e0038c6bbd34556af3f84f9e2a7c2c5 => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:35:11.845176 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8ea07c-2684-4ac7-9527-594fa1a68ac7 None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:11.916348 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540036ee-0208-49c9-8b49-083ec6548264 None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:11.951203 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:11.967096 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557e2596-b6d0-4a3a-a333-24fa8f69e6ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:11.968954 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557e2596-b6d0-4a3a-a333-24fa8f69e6ca None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:11.969460 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557e2596-b6d0-4a3a-a333-24fa8f69e6ca None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:11.970232 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557e2596-b6d0-4a3a-a333-24fa8f69e6ca None admin] PATH_INFO: `/v3/domains` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:11.971306 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:11.972777 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557e2596-b6d0-4a3a-a333-24fa8f69e6ca None admin] RBAC: Adding query filter params (name=Default) {{(pid=86827) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:35:11.973364 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557e2596-b6d0-4a3a-a333-24fa8f69e6ca None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:11.974336 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8ea07c-2684-4ac7-9527-594fa1a68ac7 None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:11.975709 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557e2596-b6d0-4a3a-a333-24fa8f69e6ca None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:11.999425 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 345/675] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 10:35:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:35:12.000203 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557e2596-b6d0-4a3a-a333-24fa8f69e6ca None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:12.019498 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925bbd92-c83d-432a-9b19-48b871809273 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:12.020124 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925bbd92-c83d-432a-9b19-48b871809273 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:12.020404 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925bbd92-c83d-432a-9b19-48b871809273 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:12.020784 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925bbd92-c83d-432a-9b19-48b871809273 None admin] PATH_INFO: `/v3/projects` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:12.021542 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925bbd92-c83d-432a-9b19-48b871809273 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86828) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:35:12.021846 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925bbd92-c83d-432a-9b19-48b871809273 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:12.023960 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925bbd92-c83d-432a-9b19-48b871809273 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:12.037330 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 331/676] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Jul 27 10:35:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:35:12.039274 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925bbd92-c83d-432a-9b19-48b871809273 None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:12.151329 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557e2596-b6d0-4a3a-a333-24fa8f69e6ca None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:12.171029 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925bbd92-c83d-432a-9b19-48b871809273 None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:12.191803 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0d3e60-836e-468f-a6f7-136c52a553dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:12.192467 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0d3e60-836e-468f-a6f7-136c52a553dd None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:12.192752 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0d3e60-836e-468f-a6f7-136c52a553dd None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:12.193032 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0d3e60-836e-468f-a6f7-136c52a553dd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:12.206784 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0d3e60-836e-468f-a6f7-136c52a553dd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:12.208246 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0d3e60-836e-468f-a6f7-136c52a553dd None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:12.218629 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcb58df-08cc-4a14-b6b3-4ff7507b26c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:12.219289 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcb58df-08cc-4a14-b6b3-4ff7507b26c6 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:12.219574 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcb58df-08cc-4a14-b6b3-4ff7507b26c6 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:12.219852 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcb58df-08cc-4a14-b6b3-4ff7507b26c6 None admin] PATH_INFO: `/v3/users/swift` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:12.223658 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application [None req-fc0d3e60-836e-468f-a6f7-136c52a553dd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 10:35:12.223658 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:35:12.223658 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:35:12.223658 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:35:12.223658 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:35:12.223658 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:35:12.223658 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:35:12.223658 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:35:12.223658 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:35:12.223658 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:35:12.223658 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:35:12.223658 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:35:12.223658 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 10:35:12.223658 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 10:35:12.223658 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 10:35:12.224502 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 10:35:12.224502 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:35:12.224502 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:35:12.224502 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:35:12.224502 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:35:12.224502 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:35:12.224502 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:35:12.224502 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:35:12.224502 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application with Lock( Jul 27 10:35:12.224502 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:35:12.224502 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:35:12.224502 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:35:12.224502 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:35:12.224502 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:35:12.224502 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:35:12.224502 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:35:12.224502 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:35:12.225804 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 10:35:12.225804 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 10:35:12.225804 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 10:35:12.225804 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Jul 27 10:35:12.225804 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 346/677] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 10:35:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 10:35:12.226084 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0d3e60-836e-468f-a6f7-136c52a553dd None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:12.227960 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfcb58df-08cc-4a14-b6b3-4ff7507b26c6 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:12.229418 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfcb58df-08cc-4a14-b6b3-4ff7507b26c6 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:12.247241 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application [None req-bfcb58df-08cc-4a14-b6b3-4ff7507b26c6 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 27 10:35:12.247241 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:35:12.247241 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:35:12.247241 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:35:12.247241 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:35:12.247241 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:35:12.247241 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:35:12.247241 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:35:12.247241 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:35:12.247241 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:35:12.247241 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:35:12.247241 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:35:12.247241 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 10:35:12.247241 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 10:35:12.247241 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 10:35:12.247241 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 10:35:12.248438 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:35:12.248438 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:35:12.248438 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 10:35:12.248438 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 10:35:12.248438 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 10:35:12.248438 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 10:35:12.248438 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:35:12.248438 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:35:12.248438 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:35:12.248438 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:35:12.248438 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:35:12.248438 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application with Lock( Jul 27 10:35:12.248438 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:35:12.248438 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:35:12.248438 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:35:12.248438 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:35:12.248438 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:35:12.248438 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:35:12.249636 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:35:12.249636 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:35:12.249636 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 10:35:12.249636 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 10:35:12.249636 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 10:35:12.249636 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 10:35:12.249636 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 10:35:12.249636 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 10:35:12.249636 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 27 10:35:12.249636 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Jul 27 10:35:12.250873 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 332/678] 10.222.0.36 () {56 vars in 1096 bytes} [Wed Jul 27 10:35:12 2022] GET /identity/v3/users/swift => generated 83 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 10:35:12.251020 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcb58df-08cc-4a14-b6b3-4ff7507b26c6 None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:12.280392 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:12.281771 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:12.369133 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcb58df-08cc-4a14-b6b3-4ff7507b26c6 None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:12.371578 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0d3e60-836e-468f-a6f7-136c52a553dd None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:12.405993 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0492c38f-9f15-4303-8d95-1ed11831424c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:12.406543 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0492c38f-9f15-4303-8d95-1ed11831424c None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:12.406900 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0492c38f-9f15-4303-8d95-1ed11831424c None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:12.407146 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0492c38f-9f15-4303-8d95-1ed11831424c None admin] PATH_INFO: `/v3/domains` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:12.407846 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0492c38f-9f15-4303-8d95-1ed11831424c None admin] RBAC: Adding query filter params (name=Default) {{(pid=86828) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:35:12.408194 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0492c38f-9f15-4303-8d95-1ed11831424c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:12.408805 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432e22b1-af1f-48b0-ba0d-0e68bb0dfe70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:12.409421 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432e22b1-af1f-48b0-ba0d-0e68bb0dfe70 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:12.409526 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0492c38f-9f15-4303-8d95-1ed11831424c None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:12.409704 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432e22b1-af1f-48b0-ba0d-0e68bb0dfe70 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:12.409977 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432e22b1-af1f-48b0-ba0d-0e68bb0dfe70 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:12.410907 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432e22b1-af1f-48b0-ba0d-0e68bb0dfe70 None admin] RBAC: Adding query filter params (role.id=9e0038c6bbd34556af3f84f9e2a7c2c5, scope.project.id=5f296d8e3b8f4001909addc49d8dac01, user.id=864d35f268cf45148756109759bb2743) {{(pid=86827) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:35:12.411112 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432e22b1-af1f-48b0-ba0d-0e68bb0dfe70 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:12.412523 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432e22b1-af1f-48b0-ba0d-0e68bb0dfe70 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:12.426674 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 347/679] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Jul 27 10:35:12 2022] GET /identity/v3/role_assignments?user.id=864d35f268cf45148756109759bb2743&scope.project.id=5f296d8e3b8f4001909addc49d8dac01&role.id=9e0038c6bbd34556af3f84f9e2a7c2c5 => generated 608 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:35:12.426674 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432e22b1-af1f-48b0-ba0d-0e68bb0dfe70 None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:12.435735 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 333/680] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 10:35:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:35:12.437609 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240e7a75-bfba-4411-8c18-fbd32260a42c None None] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:12.438042 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240e7a75-bfba-4411-8c18-fbd32260a42c None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:12.438472 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240e7a75-bfba-4411-8c18-fbd32260a42c None None] PATH_INFO: `/` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:12.440607 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 334/681] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 10:35:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 10:35:12.441601 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240e7a75-bfba-4411-8c18-fbd32260a42c None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:12.541679 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:12.551081 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:12.555418 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432e22b1-af1f-48b0-ba0d-0e68bb0dfe70 None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:12.574583 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240e7a75-bfba-4411-8c18-fbd32260a42c None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:12.596184 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbea3b96-c2ac-44b5-919c-f9556bbe96be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:12.596813 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbea3b96-c2ac-44b5-919c-f9556bbe96be None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:12.597091 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbea3b96-c2ac-44b5-919c-f9556bbe96be None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:12.597364 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbea3b96-c2ac-44b5-919c-f9556bbe96be None admin] PATH_INFO: `/v3/users` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:12.598273 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbea3b96-c2ac-44b5-919c-f9556bbe96be None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=86827) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:35:12.598583 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbea3b96-c2ac-44b5-919c-f9556bbe96be None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:12.600308 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbea3b96-c2ac-44b5-919c-f9556bbe96be None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:12.614645 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412de853-05ca-4f89-9aca-90da207d7b39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:12.615677 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412de853-05ca-4f89-9aca-90da207d7b39 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:12.616666 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412de853-05ca-4f89-9aca-90da207d7b39 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:12.617150 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412de853-05ca-4f89-9aca-90da207d7b39 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:12.620672 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 348/682] 10.222.0.36 () {56 vars in 1135 bytes} [Wed Jul 27 10:35:12 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:35:12.621999 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be903d39-a21e-4077-8ad1-4076b3633c73 None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:12.622290 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be903d39-a21e-4077-8ad1-4076b3633c73 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:12.622567 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be903d39-a21e-4077-8ad1-4076b3633c73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:12.625688 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412de853-05ca-4f89-9aca-90da207d7b39 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:12.627902 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412de853-05ca-4f89-9aca-90da207d7b39 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:12.640661 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application [None req-412de853-05ca-4f89-9aca-90da207d7b39 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 10:35:12.640661 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:35:12.640661 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:35:12.640661 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:35:12.640661 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:35:12.640661 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:35:12.640661 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:35:12.640661 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:35:12.640661 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:35:12.640661 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:35:12.640661 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:35:12.640661 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:35:12.640661 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 10:35:12.640661 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 10:35:12.640661 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 10:35:12.641621 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 10:35:12.641621 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:35:12.641621 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:35:12.641621 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:35:12.641621 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:35:12.641621 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:35:12.641621 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:35:12.641621 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:35:12.641621 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application with Lock( Jul 27 10:35:12.641621 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:35:12.641621 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:35:12.641621 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:35:12.641621 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:35:12.641621 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:35:12.641621 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:35:12.641621 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:35:12.641621 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:35:12.642506 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 10:35:12.642506 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 10:35:12.642506 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 10:35:12.642506 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 10:35:12.642506 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 10:35:12.642506 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 10:35:12.642506 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 10:35:12.642506 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Jul 27 10:35:12.642506 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 335/683] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Jul 27 10:35:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 10:35:12.643048 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-be903d39-a21e-4077-8ad1-4076b3633c73 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:35:12.643130 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412de853-05ca-4f89-9aca-90da207d7b39 None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:12.644310 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-be903d39-a21e-4077-8ad1-4076b3633c73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:35:12.719199 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 349/684] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 10:35:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 10:35:12.720650 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be903d39-a21e-4077-8ad1-4076b3633c73 None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:12.764724 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412de853-05ca-4f89-9aca-90da207d7b39 None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:12.800869 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2808068-6a73-48ec-aae4-e41beaea4cf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:12.801508 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2808068-6a73-48ec-aae4-e41beaea4cf4 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:12.801792 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2808068-6a73-48ec-aae4-e41beaea4cf4 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:12.802070 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2808068-6a73-48ec-aae4-e41beaea4cf4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:12.813793 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:12.823713 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:12.840973 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be903d39-a21e-4077-8ad1-4076b3633c73 None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:12.873988 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2808068-6a73-48ec-aae4-e41beaea4cf4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:12.876023 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2808068-6a73-48ec-aae4-e41beaea4cf4 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:12.877786 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80c55e1-3824-436c-80f9-66f3ab99297e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:12.878509 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80c55e1-3824-436c-80f9-66f3ab99297e None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:12.878820 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80c55e1-3824-436c-80f9-66f3ab99297e None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:12.879190 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80c55e1-3824-436c-80f9-66f3ab99297e None admin] PATH_INFO: `/v3/projects` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:12.879951 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80c55e1-3824-436c-80f9-66f3ab99297e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86827) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:35:12.880258 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80c55e1-3824-436c-80f9-66f3ab99297e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:12.882291 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80c55e1-3824-436c-80f9-66f3ab99297e None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:12.900282 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 350/685] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Jul 27 10:35:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:35:12.901649 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80c55e1-3824-436c-80f9-66f3ab99297e None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:12.994189 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 336/686] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jul 27 10:35:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 10:35:12.995048 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2808068-6a73-48ec-aae4-e41beaea4cf4 None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:13.025395 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80c55e1-3824-436c-80f9-66f3ab99297e None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:13.037326 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:13.066094 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1a72a0-2c53-4ddb-a01b-8dcafc8a7067 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:13.067037 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1a72a0-2c53-4ddb-a01b-8dcafc8a7067 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:13.067594 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1a72a0-2c53-4ddb-a01b-8dcafc8a7067 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:13.068019 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1a72a0-2c53-4ddb-a01b-8dcafc8a7067 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:13.079981 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1a72a0-2c53-4ddb-a01b-8dcafc8a7067 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:13.081785 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1a72a0-2c53-4ddb-a01b-8dcafc8a7067 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:13.088836 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:13.099472 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application [None req-1e1a72a0-2c53-4ddb-a01b-8dcafc8a7067 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 10:35:13.099472 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:35:13.099472 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:35:13.099472 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:35:13.099472 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:35:13.099472 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:35:13.099472 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:35:13.099472 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:35:13.099472 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:35:13.099472 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:35:13.099472 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:35:13.099472 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:35:13.099472 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 10:35:13.099472 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 10:35:13.099472 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 10:35:13.101031 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 10:35:13.101031 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:35:13.101031 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:35:13.101031 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:35:13.101031 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:35:13.101031 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:35:13.101031 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:35:13.101031 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:35:13.101031 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application with Lock( Jul 27 10:35:13.101031 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:35:13.101031 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:35:13.101031 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:35:13.101031 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:35:13.101031 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:35:13.101031 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:35:13.101031 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:35:13.101031 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:35:13.102129 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 10:35:13.102129 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 10:35:13.102129 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 10:35:13.102129 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Jul 27 10:35:13.102396 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2808068-6a73-48ec-aae4-e41beaea4cf4 None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:13.108984 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 351/687] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 10:35:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 10:35:13.110045 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1a72a0-2c53-4ddb-a01b-8dcafc8a7067 None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:13.137182 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c7def1-b812-4326-ba43-31eb0eec5398 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:13.137815 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c7def1-b812-4326-ba43-31eb0eec5398 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:13.138106 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c7def1-b812-4326-ba43-31eb0eec5398 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:13.138383 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c7def1-b812-4326-ba43-31eb0eec5398 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:13.139230 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c7def1-b812-4326-ba43-31eb0eec5398 None admin] RBAC: Adding query filter params (role.id=9e0038c6bbd34556af3f84f9e2a7c2c5, scope.project.id=5f296d8e3b8f4001909addc49d8dac01, user.id=c7922d0a1a7c4e808d0b2a80dca0da40) {{(pid=86828) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:35:13.139544 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c7def1-b812-4326-ba43-31eb0eec5398 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:13.140972 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c7def1-b812-4326-ba43-31eb0eec5398 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:13.147462 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 337/688] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Jul 27 10:35:12 2022] GET /identity/v3/role_assignments?user.id=c7922d0a1a7c4e808d0b2a80dca0da40&scope.project.id=5f296d8e3b8f4001909addc49d8dac01&role.id=9e0038c6bbd34556af3f84f9e2a7c2c5 => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:35:13.148277 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c7def1-b812-4326-ba43-31eb0eec5398 None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:13.240811 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1a72a0-2c53-4ddb-a01b-8dcafc8a7067 None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:13.264619 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c7def1-b812-4326-ba43-31eb0eec5398 None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:13.272794 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:13.280300 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8539dda3-599e-4e50-97c0-72ab04d0b944 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:13.281247 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8539dda3-599e-4e50-97c0-72ab04d0b944 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:13.281764 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8539dda3-599e-4e50-97c0-72ab04d0b944 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:13.282107 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8539dda3-599e-4e50-97c0-72ab04d0b944 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:13.295092 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8539dda3-599e-4e50-97c0-72ab04d0b944 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:13.296661 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8539dda3-599e-4e50-97c0-72ab04d0b944 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:13.304727 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db90ddc-5519-4019-b099-49da660e0c81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:13.305732 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db90ddc-5519-4019-b099-49da660e0c81 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:13.306209 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db90ddc-5519-4019-b099-49da660e0c81 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:13.306700 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db90ddc-5519-4019-b099-49da660e0c81 None admin] PATH_INFO: `/v3/domains` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:13.307688 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db90ddc-5519-4019-b099-49da660e0c81 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86828) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:35:13.308204 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db90ddc-5519-4019-b099-49da660e0c81 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:13.309823 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db90ddc-5519-4019-b099-49da660e0c81 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:13.313349 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application [None req-8539dda3-599e-4e50-97c0-72ab04d0b944 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 10:35:13.313349 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:35:13.313349 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:35:13.313349 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:35:13.313349 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:35:13.313349 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:35:13.313349 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:35:13.313349 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:35:13.313349 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:35:13.313349 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:35:13.313349 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:35:13.313349 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:35:13.313349 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 10:35:13.313349 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 10:35:13.313349 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 10:35:13.314212 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 10:35:13.314212 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:35:13.314212 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:35:13.314212 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:35:13.314212 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:35:13.314212 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:35:13.314212 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:35:13.314212 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:35:13.314212 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application with Lock( Jul 27 10:35:13.314212 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:35:13.314212 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:35:13.314212 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:35:13.314212 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:35:13.314212 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:35:13.314212 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:35:13.314212 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:35:13.314212 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:35:13.315131 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 10:35:13.315131 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 10:35:13.315131 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 10:35:13.315131 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Jul 27 10:35:13.319108 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 352/689] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 10:35:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 10:35:13.321017 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8539dda3-599e-4e50-97c0-72ab04d0b944 None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:13.326790 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 338/690] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 10:35:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:35:13.340066 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db90ddc-5519-4019-b099-49da660e0c81 None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:13.404299 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:13.483113 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8539dda3-599e-4e50-97c0-72ab04d0b944 None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:13.484778 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db90ddc-5519-4019-b099-49da660e0c81 None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:13.536767 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f15f916-d49c-4269-b057-38fcac1a7a15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:13.537325 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f15f916-d49c-4269-b057-38fcac1a7a15 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:13.537608 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f15f916-d49c-4269-b057-38fcac1a7a15 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:13.538439 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f15f916-d49c-4269-b057-38fcac1a7a15 None admin] PATH_INFO: `/v3/domains` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:13.539166 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f15f916-d49c-4269-b057-38fcac1a7a15 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86827) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:35:13.539472 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f15f916-d49c-4269-b057-38fcac1a7a15 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:13.540842 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f15f916-d49c-4269-b057-38fcac1a7a15 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:13.544740 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24896684-c2b5-4c38-9909-339a9951ac57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:13.545370 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24896684-c2b5-4c38-9909-339a9951ac57 None admin] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:13.545650 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24896684-c2b5-4c38-9909-339a9951ac57 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:13.545923 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24896684-c2b5-4c38-9909-339a9951ac57 None admin] PATH_INFO: `/v3/users` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:13.546632 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24896684-c2b5-4c38-9909-339a9951ac57 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:13.548078 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24896684-c2b5-4c38-9909-339a9951ac57 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:13.562080 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 353/691] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 10:35:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:35:13.576438 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f15f916-d49c-4269-b057-38fcac1a7a15 None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:13.579950 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 339/692] 10.222.0.36 () {60 vars in 1132 bytes} [Wed Jul 27 10:35:13 2022] POST /identity/v3/users => generated 254 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 10:35:13.698701 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f15f916-d49c-4269-b057-38fcac1a7a15 None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:13.722979 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:13.737211 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03dc4a0b-1e76-4ece-a7d8-1828e286e9f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:13.738215 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dc4a0b-1e76-4ece-a7d8-1828e286e9f0 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:13.738762 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dc4a0b-1e76-4ece-a7d8-1828e286e9f0 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:13.739238 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dc4a0b-1e76-4ece-a7d8-1828e286e9f0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:13.747655 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03dc4a0b-1e76-4ece-a7d8-1828e286e9f0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:13.749653 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03dc4a0b-1e76-4ece-a7d8-1828e286e9f0 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:13.763454 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application [None req-03dc4a0b-1e76-4ece-a7d8-1828e286e9f0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 10:35:13.763454 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:35:13.763454 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:35:13.763454 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:35:13.763454 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:35:13.763454 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:35:13.763454 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:35:13.763454 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:35:13.763454 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:35:13.763454 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:35:13.763454 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:35:13.763454 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:35:13.763454 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 10:35:13.763454 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 10:35:13.763454 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 10:35:13.764516 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 10:35:13.764516 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:35:13.764516 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:35:13.764516 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:35:13.764516 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:35:13.764516 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:35:13.764516 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:35:13.764516 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:35:13.764516 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application with Lock( Jul 27 10:35:13.764516 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:35:13.764516 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:35:13.764516 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:35:13.764516 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:35:13.764516 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:35:13.764516 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:35:13.764516 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:35:13.764516 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:35:13.765420 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 10:35:13.765420 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 10:35:13.765420 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 10:35:13.765420 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 10:35:13.765420 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 10:35:13.765420 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 10:35:13.765420 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 10:35:13.765420 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Jul 27 10:35:13.768043 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 354/693] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Jul 27 10:35:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 10:35:13.774395 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24896684-c2b5-4c38-9909-339a9951ac57 None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:13.809151 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:13.898388 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24896684-c2b5-4c38-9909-339a9951ac57 None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:13.935835 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a0501c-ed26-4621-aece-19a20f8e92db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:13.936467 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a0501c-ed26-4621-aece-19a20f8e92db None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:13.936753 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a0501c-ed26-4621-aece-19a20f8e92db None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:13.937031 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a0501c-ed26-4621-aece-19a20f8e92db None admin] PATH_INFO: `/v3/projects` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:13.937796 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a0501c-ed26-4621-aece-19a20f8e92db None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86828) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:35:13.938107 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a0501c-ed26-4621-aece-19a20f8e92db None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:13.939599 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a0501c-ed26-4621-aece-19a20f8e92db None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:13.955759 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 340/694] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Jul 27 10:35:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:35:13.961910 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03dc4a0b-1e76-4ece-a7d8-1828e286e9f0 None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:14.068651 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03dc4a0b-1e76-4ece-a7d8-1828e286e9f0 None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:14.106191 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b655b1-f761-4c49-be44-f6b13371148f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:14.106852 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b655b1-f761-4c49-be44-f6b13371148f None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:14.107136 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b655b1-f761-4c49-be44-f6b13371148f None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:14.107417 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b655b1-f761-4c49-be44-f6b13371148f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:14.108213 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b655b1-f761-4c49-be44-f6b13371148f None admin] RBAC: Adding query filter params (role.id=9e0038c6bbd34556af3f84f9e2a7c2c5, scope.project.id=5f296d8e3b8f4001909addc49d8dac01, user.id=89ac506eefcd4e42a84b44fa15fb3975) {{(pid=86827) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:35:14.108527 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b655b1-f761-4c49-be44-f6b13371148f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:14.109961 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b655b1-f761-4c49-be44-f6b13371148f None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:14.117031 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 355/695] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Jul 27 10:35:13 2022] GET /identity/v3/role_assignments?user.id=89ac506eefcd4e42a84b44fa15fb3975&scope.project.id=5f296d8e3b8f4001909addc49d8dac01&role.id=9e0038c6bbd34556af3f84f9e2a7c2c5 => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 10:35:14.396857 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b26949-df76-4e2e-bcaf-80ee3aada32a None None] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:14.397388 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b26949-df76-4e2e-bcaf-80ee3aada32a None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:14.397481 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b26949-df76-4e2e-bcaf-80ee3aada32a None None] PATH_INFO: `/` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:14.399098 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 341/696] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 10:35:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 10:35:14.404790 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6201d6-2e59-48c1-8137-40a97b8bcc38 None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:14.405328 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6201d6-2e59-48c1-8137-40a97b8bcc38 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:14.405800 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6201d6-2e59-48c1-8137-40a97b8bcc38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:14.426220 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-aa6201d6-2e59-48c1-8137-40a97b8bcc38 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:35:14.427443 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-aa6201d6-2e59-48c1-8137-40a97b8bcc38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:35:14.456710 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:14.508523 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 356/697] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 10:35:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 10:35:14.532351 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b26949-df76-4e2e-bcaf-80ee3aada32a None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:14.624723 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:14.646092 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b26949-df76-4e2e-bcaf-80ee3aada32a None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:14.682881 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31d54d0-e451-496e-93c9-7ffc5b8a6e46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:14.683868 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31d54d0-e451-496e-93c9-7ffc5b8a6e46 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:14.684346 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31d54d0-e451-496e-93c9-7ffc5b8a6e46 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:14.684828 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31d54d0-e451-496e-93c9-7ffc5b8a6e46 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:14.695530 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31d54d0-e451-496e-93c9-7ffc5b8a6e46 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 10:35:14.700927 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application [None req-b31d54d0-e451-496e-93c9-7ffc5b8a6e46 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 10:35:14.700927 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:35:14.700927 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:35:14.700927 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:35:14.700927 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:35:14.700927 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:35:14.700927 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:35:14.700927 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:35:14.700927 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:35:14.700927 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:35:14.700927 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:35:14.700927 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:35:14.700927 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 10:35:14.700927 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 10:35:14.700927 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 10:35:14.700927 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 10:35:14.701854 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 10:35:14.701854 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 10:35:14.701854 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 10:35:14.701854 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 10:35:14.701854 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:35:14.701854 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:35:14.701854 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:35:14.701854 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:35:14.701854 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:35:14.701854 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:35:14.701854 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:35:14.701854 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application with Lock( Jul 27 10:35:14.701854 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:35:14.701854 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:35:14.701854 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:35:14.701854 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:35:14.702716 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:35:14.702716 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:35:14.702716 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:35:14.702716 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:35:14.702716 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 10:35:14.702716 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 10:35:14.702716 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 10:35:14.702716 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 10:35:14.702716 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 10:35:14.702716 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 10:35:14.702716 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 10:35:14.702716 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Jul 27 10:35:14.709872 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 342/698] 10.222.0.36 () {56 vars in 1096 bytes} [Wed Jul 27 10:35:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 10:35:14.711702 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6201d6-2e59-48c1-8137-40a97b8bcc38 None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:14.738410 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af6c500-9487-46b8-9b01-488a9636f241 None None] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:14.739071 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af6c500-9487-46b8-9b01-488a9636f241 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:14.739576 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af6c500-9487-46b8-9b01-488a9636f241 None None] PATH_INFO: `/` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:14.742026 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 343/699] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 10:35:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 10:35:14.751084 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1272ac-2248-480d-969b-b217056de115 None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:14.751651 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1272ac-2248-480d-969b-b217056de115 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:14.752137 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1272ac-2248-480d-969b-b217056de115 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:14.771510 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-cc1272ac-2248-480d-969b-b217056de115 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:35:14.773140 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-cc1272ac-2248-480d-969b-b217056de115 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:35:14.861640 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6201d6-2e59-48c1-8137-40a97b8bcc38 None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:14.920152 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 344/700] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 10:35:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 10:35:14.955602 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1272ac-2248-480d-969b-b217056de115 None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:14.962164 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e57c72-9075-4085-8147-faa11e5807c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:14.962888 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e57c72-9075-4085-8147-faa11e5807c0 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:14.963176 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e57c72-9075-4085-8147-faa11e5807c0 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:14.963455 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e57c72-9075-4085-8147-faa11e5807c0 None admin] PATH_INFO: `/v3/roles` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:14.964251 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e57c72-9075-4085-8147-faa11e5807c0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86827) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:35:14.964566 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e57c72-9075-4085-8147-faa11e5807c0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:14.966099 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e57c72-9075-4085-8147-faa11e5807c0 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:14.975427 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 357/701] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jul 27 10:35:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:35:14.983916 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e57c72-9075-4085-8147-faa11e5807c0 None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:15.039508 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:15.054859 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:15.075498 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1272ac-2248-480d-969b-b217056de115 None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:15.112112 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fad038d-340d-4004-8a99-64837c219ab3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:15.113161 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fad038d-340d-4004-8a99-64837c219ab3 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:15.113648 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fad038d-340d-4004-8a99-64837c219ab3 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:15.114131 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fad038d-340d-4004-8a99-64837c219ab3 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:15.115010 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fad038d-340d-4004-8a99-64837c219ab3 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:15.117120 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fad038d-340d-4004-8a99-64837c219ab3 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:15.128699 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application [None req-1fad038d-340d-4004-8a99-64837c219ab3 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 27 10:35:15.128699 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:35:15.128699 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 27 10:35:15.128699 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 27 10:35:15.128699 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 27 10:35:15.128699 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 27 10:35:15.128699 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 27 10:35:15.128699 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 10:35:15.128699 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 27 10:35:15.128699 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Jul 27 10:35:15.128699 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 10:35:15.128699 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Jul 27 10:35:15.128699 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:35:15.128699 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:35:15.128699 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:35:15.128699 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:35:15.129683 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:35:15.129683 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:35:15.129683 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:35:15.129683 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:35:15.129683 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:35:15.129683 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:35:15.129683 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:35:15.129683 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 27 10:35:15.129683 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 27 10:35:15.129683 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 27 10:35:15.129683 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 27 10:35:15.129683 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:35:15.129683 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:35:15.129683 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:35:15.129683 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:35:15.129683 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:35:15.130513 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:35:15.130513 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:35:15.130513 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application with Lock( Jul 27 10:35:15.130513 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:35:15.130513 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:35:15.130513 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:35:15.130513 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:35:15.130513 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:35:15.130513 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:35:15.130513 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:35:15.130513 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:35:15.130513 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 27 10:35:15.130513 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 10:35:15.130513 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 27 10:35:15.130513 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Jul 27 10:35:15.134283 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 345/702] 10.222.0.36 () {56 vars in 1129 bytes} [Wed Jul 27 10:35:14 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 10:35:15.139893 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fad038d-340d-4004-8a99-64837c219ab3 None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:15.172031 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e57c72-9075-4085-8147-faa11e5807c0 None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:15.209170 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc08957-e03c-4459-a348-5ee5734eb35f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:15.210199 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc08957-e03c-4459-a348-5ee5734eb35f None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:15.210724 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc08957-e03c-4459-a348-5ee5734eb35f None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:15.211311 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc08957-e03c-4459-a348-5ee5734eb35f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:15.223425 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc08957-e03c-4459-a348-5ee5734eb35f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:15.227558 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc08957-e03c-4459-a348-5ee5734eb35f None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:15.257156 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application [None req-1fc08957-e03c-4459-a348-5ee5734eb35f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 10:35:15.257156 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:35:15.257156 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:35:15.257156 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:35:15.257156 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:35:15.257156 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:35:15.257156 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:35:15.257156 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:35:15.257156 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:35:15.257156 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:35:15.257156 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:35:15.257156 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:35:15.257156 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 10:35:15.257156 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 10:35:15.257156 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 10:35:15.258098 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 10:35:15.258098 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:35:15.258098 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:35:15.258098 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:35:15.258098 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:35:15.258098 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:35:15.258098 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:35:15.258098 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:35:15.258098 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application with Lock( Jul 27 10:35:15.258098 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:35:15.258098 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:35:15.258098 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:35:15.258098 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:35:15.258098 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:35:15.258098 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:35:15.258098 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:35:15.258098 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:35:15.259340 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 10:35:15.259340 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 10:35:15.259340 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 10:35:15.259340 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Jul 27 10:35:15.261245 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 358/703] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 10:35:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 10:35:15.266746 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc08957-e03c-4459-a348-5ee5734eb35f None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:15.331807 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fad038d-340d-4004-8a99-64837c219ab3 None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:15.368443 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9625e53b-6f1e-46c2-94e4-d1e427769ac4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:15.369436 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9625e53b-6f1e-46c2-94e4-d1e427769ac4 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:15.369916 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9625e53b-6f1e-46c2-94e4-d1e427769ac4 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:15.370385 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9625e53b-6f1e-46c2-94e4-d1e427769ac4 None admin] PATH_INFO: `/v3/services` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:15.371279 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9625e53b-6f1e-46c2-94e4-d1e427769ac4 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=86828) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:35:15.371801 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9625e53b-6f1e-46c2-94e4-d1e427769ac4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:15.373356 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9625e53b-6f1e-46c2-94e4-d1e427769ac4 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:15.374694 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:15.379602 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 346/704] 10.222.0.36 () {56 vars in 1124 bytes} [Wed Jul 27 10:35:15 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 10:35:15.388024 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9625e53b-6f1e-46c2-94e4-d1e427769ac4 None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:15.443204 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:15.478217 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc08957-e03c-4459-a348-5ee5734eb35f None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:15.517571 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c67138-64ee-4b80-acdc-24c49ae832aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:15.518332 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c67138-64ee-4b80-acdc-24c49ae832aa None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:15.518542 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c67138-64ee-4b80-acdc-24c49ae832aa None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:15.518844 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c67138-64ee-4b80-acdc-24c49ae832aa None admin] PATH_INFO: `/v3/domains` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:15.519545 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c67138-64ee-4b80-acdc-24c49ae832aa None admin] RBAC: Adding query filter params (name=Default) {{(pid=86827) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:35:15.519850 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c67138-64ee-4b80-acdc-24c49ae832aa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:15.521219 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c67138-64ee-4b80-acdc-24c49ae832aa None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:15.546462 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 359/705] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 10:35:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:35:15.552071 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c67138-64ee-4b80-acdc-24c49ae832aa None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:15.573815 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9625e53b-6f1e-46c2-94e4-d1e427769ac4 None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:15.614571 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44753e11-ebaa-499c-9881-b9212882e13b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:15.615498 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44753e11-ebaa-499c-9881-b9212882e13b None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:15.615814 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44753e11-ebaa-499c-9881-b9212882e13b None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:15.616098 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44753e11-ebaa-499c-9881-b9212882e13b None admin] PATH_INFO: `/v3/services` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:15.617034 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44753e11-ebaa-499c-9881-b9212882e13b None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=86828) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:35:15.617342 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44753e11-ebaa-499c-9881-b9212882e13b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:15.618762 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44753e11-ebaa-499c-9881-b9212882e13b None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:15.626082 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 347/706] 10.222.0.36 () {56 vars in 1124 bytes} [Wed Jul 27 10:35:15 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:35:15.746294 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c67138-64ee-4b80-acdc-24c49ae832aa None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:15.800174 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72746d34-4172-49ce-9f92-e6f4db690c84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:15.800911 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72746d34-4172-49ce-9f92-e6f4db690c84 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:15.801245 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72746d34-4172-49ce-9f92-e6f4db690c84 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:15.801527 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72746d34-4172-49ce-9f92-e6f4db690c84 None admin] PATH_INFO: `/v3/users/nova` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:15.811773 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72746d34-4172-49ce-9f92-e6f4db690c84 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:15.813241 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72746d34-4172-49ce-9f92-e6f4db690c84 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:15.815234 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:15.826941 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application [None req-72746d34-4172-49ce-9f92-e6f4db690c84 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 27 10:35:15.826941 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:35:15.826941 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:35:15.826941 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:35:15.826941 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:35:15.826941 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:35:15.826941 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:35:15.826941 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:35:15.826941 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:35:15.826941 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:35:15.826941 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:35:15.826941 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:35:15.826941 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 10:35:15.826941 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 10:35:15.826941 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 10:35:15.826941 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 10:35:15.827913 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:35:15.827913 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:35:15.827913 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 10:35:15.827913 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 10:35:15.827913 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 10:35:15.827913 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 10:35:15.827913 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:35:15.827913 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:35:15.827913 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:35:15.827913 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:35:15.827913 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:35:15.827913 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application with Lock( Jul 27 10:35:15.827913 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:35:15.827913 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:35:15.827913 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:35:15.827913 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:35:15.827913 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:35:15.827913 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:35:15.829553 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:35:15.829553 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:35:15.829553 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 10:35:15.829553 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 10:35:15.829553 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 10:35:15.829553 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 10:35:15.829553 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 10:35:15.829553 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 10:35:15.829553 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 27 10:35:15.829553 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Jul 27 10:35:15.829553 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 360/707] 10.222.0.36 () {56 vars in 1093 bytes} [Wed Jul 27 10:35:15 2022] GET /identity/v3/users/nova => generated 82 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 10:35:15.835652 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44753e11-ebaa-499c-9881-b9212882e13b None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:15.915727 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:15.925703 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258530a1-dd4e-46ba-9211-dde6cae96a94 None None] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:15.925999 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258530a1-dd4e-46ba-9211-dde6cae96a94 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:15.926273 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258530a1-dd4e-46ba-9211-dde6cae96a94 None None] PATH_INFO: `/` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:15.934711 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 361/708] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 10:35:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 10:35:15.936198 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b53e50f-323c-4d6b-a2f3-d3c380afc0f7 None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:15.936487 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b53e50f-323c-4d6b-a2f3-d3c380afc0f7 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:15.936773 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b53e50f-323c-4d6b-a2f3-d3c380afc0f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:15.978257 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44753e11-ebaa-499c-9881-b9212882e13b None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:15.979622 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-6b53e50f-323c-4d6b-a2f3-d3c380afc0f7 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:35:15.980823 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-6b53e50f-323c-4d6b-a2f3-d3c380afc0f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:35:16.014579 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c596bc8-d326-45b6-ab2d-252d9685b39a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:16.015160 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c596bc8-d326-45b6-ab2d-252d9685b39a None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:16.015443 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c596bc8-d326-45b6-ab2d-252d9685b39a None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:16.015726 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c596bc8-d326-45b6-ab2d-252d9685b39a None admin] PATH_INFO: `/v3/users` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:16.016534 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c596bc8-d326-45b6-ab2d-252d9685b39a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86828) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:35:16.016840 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c596bc8-d326-45b6-ab2d-252d9685b39a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:16.018261 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c596bc8-d326-45b6-ab2d-252d9685b39a None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:16.037505 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 348/709] 10.222.0.36 () {56 vars in 1133 bytes} [Wed Jul 27 10:35:15 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:35:16.042966 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c596bc8-d326-45b6-ab2d-252d9685b39a None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:16.116103 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 362/710] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 10:35:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 27 10:35:16.161094 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b53e50f-323c-4d6b-a2f3-d3c380afc0f7 None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:16.198152 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c596bc8-d326-45b6-ab2d-252d9685b39a None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:16.225065 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:16.240435 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6467517-a54f-4c56-83d5-323c8c8b5c17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:16.241109 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6467517-a54f-4c56-83d5-323c8c8b5c17 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:16.241395 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6467517-a54f-4c56-83d5-323c8c8b5c17 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:16.241674 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6467517-a54f-4c56-83d5-323c8c8b5c17 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:16.279596 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b53e50f-323c-4d6b-a2f3-d3c380afc0f7 None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:16.310353 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:16.317527 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6467517-a54f-4c56-83d5-323c8c8b5c17 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:16.319980 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6467517-a54f-4c56-83d5-323c8c8b5c17 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:16.321695 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bcdc40-0183-447f-b1c4-c95b9219dd67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:16.322848 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bcdc40-0183-447f-b1c4-c95b9219dd67 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:16.323357 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bcdc40-0183-447f-b1c4-c95b9219dd67 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:16.323860 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bcdc40-0183-447f-b1c4-c95b9219dd67 None admin] PATH_INFO: `/v3/services/network` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:16.324712 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bcdc40-0183-447f-b1c4-c95b9219dd67 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:16.326269 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bcdc40-0183-447f-b1c4-c95b9219dd67 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:16.339214 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application [None req-d1bcdc40-0183-447f-b1c4-c95b9219dd67 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 27 10:35:16.339214 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:35:16.339214 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 27 10:35:16.339214 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 27 10:35:16.339214 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 27 10:35:16.339214 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 27 10:35:16.339214 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 27 10:35:16.339214 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 10:35:16.339214 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 27 10:35:16.339214 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Jul 27 10:35:16.339214 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 10:35:16.339214 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Jul 27 10:35:16.339214 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:35:16.339214 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:35:16.339214 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:35:16.339214 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:35:16.339214 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:35:16.340246 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:35:16.340246 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:35:16.340246 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:35:16.340246 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:35:16.340246 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:35:16.340246 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:35:16.340246 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 27 10:35:16.340246 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 27 10:35:16.340246 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 27 10:35:16.340246 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 27 10:35:16.340246 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:35:16.340246 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:35:16.340246 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:35:16.340246 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:35:16.340246 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:35:16.340246 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:35:16.341108 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:35:16.341108 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application with Lock( Jul 27 10:35:16.341108 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:35:16.341108 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:35:16.341108 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:35:16.341108 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:35:16.341108 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:35:16.341108 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:35:16.341108 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:35:16.341108 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:35:16.341108 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 27 10:35:16.341108 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 10:35:16.341108 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 27 10:35:16.341108 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Jul 27 10:35:16.344047 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 363/711] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Jul 27 10:35:16 2022] GET /identity/v3/services/network => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 10:35:16.345507 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4be475-6333-4779-825c-d6917333c573 None None] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:16.345998 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4be475-6333-4779-825c-d6917333c573 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:16.346495 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4be475-6333-4779-825c-d6917333c573 None None] PATH_INFO: `/` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:16.348262 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 364/712] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 10:35:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 10:35:16.358724 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a142774c-acd8-4174-afee-5db344c91290 None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:16.358724 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a142774c-acd8-4174-afee-5db344c91290 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:16.358724 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a142774c-acd8-4174-afee-5db344c91290 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:16.436156 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 349/713] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jul 27 10:35:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 10:35:16.437095 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6467517-a54f-4c56-83d5-323c8c8b5c17 None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:16.463988 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:16.493925 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-a142774c-acd8-4174-afee-5db344c91290 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:35:16.495258 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-a142774c-acd8-4174-afee-5db344c91290 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:35:16.544254 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6467517-a54f-4c56-83d5-323c8c8b5c17 None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:16.580589 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af48e74c-7c90-43d4-ae94-ceb33c6069ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:16.581211 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af48e74c-7c90-43d4-ae94-ceb33c6069ec None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:16.581485 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af48e74c-7c90-43d4-ae94-ceb33c6069ec None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:16.581760 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af48e74c-7c90-43d4-ae94-ceb33c6069ec None admin] PATH_INFO: `/v3/services` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:16.582438 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af48e74c-7c90-43d4-ae94-ceb33c6069ec None admin] RBAC: Adding query filter params (name=network) {{(pid=86828) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:35:16.582822 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af48e74c-7c90-43d4-ae94-ceb33c6069ec None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:16.582924 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 365/714] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 10:35:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 27 10:35:16.583745 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a142774c-acd8-4174-afee-5db344c91290 None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:16.584224 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af48e74c-7c90-43d4-ae94-ceb33c6069ec None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:16.601253 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 350/715] 10.222.0.36 () {56 vars in 1112 bytes} [Wed Jul 27 10:35:16 2022] GET /identity/v3/services?name=network => generated 124 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:35:16.606753 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af48e74c-7c90-43d4-ae94-ceb33c6069ec None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:16.704010 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:16.721839 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a142774c-acd8-4174-afee-5db344c91290 None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:16.722173 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af48e74c-7c90-43d4-ae94-ceb33c6069ec None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:16.759442 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:16.761905 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75aaf2ac-bded-4385-a95b-aa182e762c0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:16.762623 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75aaf2ac-bded-4385-a95b-aa182e762c0e None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:16.762975 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75aaf2ac-bded-4385-a95b-aa182e762c0e None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:16.763321 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75aaf2ac-bded-4385-a95b-aa182e762c0e None admin] PATH_INFO: `/v3/services` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:16.764093 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75aaf2ac-bded-4385-a95b-aa182e762c0e None admin] RBAC: Adding query filter params (type=network) {{(pid=86828) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:35:16.764402 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75aaf2ac-bded-4385-a95b-aa182e762c0e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:16.765782 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75aaf2ac-bded-4385-a95b-aa182e762c0e None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:16.777489 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 351/716] 10.222.0.36 () {56 vars in 1112 bytes} [Wed Jul 27 10:35:16 2022] GET /identity/v3/services?type=network => generated 124 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 10:35:16.778624 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75aaf2ac-bded-4385-a95b-aa182e762c0e None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:16.779400 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d0df45-bf27-4728-b412-8dc3756ce6f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:16.780228 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d0df45-bf27-4728-b412-8dc3756ce6f4 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:16.780629 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d0df45-bf27-4728-b412-8dc3756ce6f4 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:16.780975 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d0df45-bf27-4728-b412-8dc3756ce6f4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:16.798058 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d0df45-bf27-4728-b412-8dc3756ce6f4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:16.799620 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d0df45-bf27-4728-b412-8dc3756ce6f4 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:16.818390 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application [None req-a7d0df45-bf27-4728-b412-8dc3756ce6f4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 10:35:16.818390 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:35:16.818390 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:35:16.818390 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:35:16.818390 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:35:16.818390 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:35:16.818390 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:35:16.818390 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:35:16.818390 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:35:16.818390 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:35:16.818390 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:35:16.818390 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:35:16.818390 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 10:35:16.818390 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 10:35:16.818390 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 10:35:16.820920 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 10:35:16.820920 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:35:16.820920 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:35:16.820920 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:35:16.820920 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:35:16.820920 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:35:16.820920 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:35:16.820920 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:35:16.820920 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application with Lock( Jul 27 10:35:16.820920 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:35:16.820920 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:35:16.820920 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:35:16.820920 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:35:16.820920 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:35:16.820920 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:35:16.820920 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:35:16.820920 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:35:16.822432 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 10:35:16.822432 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 10:35:16.822432 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 10:35:16.822432 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Jul 27 10:35:16.822432 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 366/717] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 10:35:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 10:35:16.828842 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d0df45-bf27-4728-b412-8dc3756ce6f4 None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:16.973320 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d0df45-bf27-4728-b412-8dc3756ce6f4 None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:16.974302 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75aaf2ac-bded-4385-a95b-aa182e762c0e None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:17.015476 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7e8601-d5f5-43fd-89aa-69994e13ef55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:17.016524 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7e8601-d5f5-43fd-89aa-69994e13ef55 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:17.017034 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7e8601-d5f5-43fd-89aa-69994e13ef55 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:17.017555 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7e8601-d5f5-43fd-89aa-69994e13ef55 None admin] PATH_INFO: `/v3/domains` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:17.018496 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7e8601-d5f5-43fd-89aa-69994e13ef55 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86827) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:35:17.018736 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9851861-b7fe-47fa-b735-0c18658e819a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:17.019270 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7e8601-d5f5-43fd-89aa-69994e13ef55 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:17.019396 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9851861-b7fe-47fa-b735-0c18658e819a None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:17.019618 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9851861-b7fe-47fa-b735-0c18658e819a None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:17.019892 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9851861-b7fe-47fa-b735-0c18658e819a None admin] PATH_INFO: `/v3/roles/service` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:17.021421 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7e8601-d5f5-43fd-89aa-69994e13ef55 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:17.034040 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9851861-b7fe-47fa-b735-0c18658e819a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 10:35:17.034979 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 367/718] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 10:35:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 10:35:17.036466 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124f6a18-bb96-4484-a578-54f307cd9bfc None None] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:17.036985 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124f6a18-bb96-4484-a578-54f307cd9bfc None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:17.037482 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124f6a18-bb96-4484-a578-54f307cd9bfc None None] PATH_INFO: `/` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:17.039320 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 368/719] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 10:35:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 10:35:17.039630 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application [None req-d9851861-b7fe-47fa-b735-0c18658e819a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 27 10:35:17.039630 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:35:17.039630 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:35:17.039630 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:35:17.039630 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:35:17.039630 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:35:17.039630 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:35:17.039630 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:35:17.039630 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:35:17.039630 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:35:17.039630 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:35:17.039630 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:35:17.039630 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 10:35:17.039630 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 10:35:17.039630 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 10:35:17.039630 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 10:35:17.040559 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 10:35:17.040559 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 10:35:17.040559 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 10:35:17.040559 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 10:35:17.040559 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:35:17.040559 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:35:17.040559 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:35:17.040559 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:35:17.040559 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:35:17.040559 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:35:17.040559 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:35:17.040559 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application with Lock( Jul 27 10:35:17.040559 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:35:17.040559 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:35:17.040559 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:35:17.040559 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:35:17.041435 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:35:17.041435 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:35:17.041435 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:35:17.041435 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:35:17.041435 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 10:35:17.041435 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 10:35:17.041435 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 10:35:17.041435 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 10:35:17.041435 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 10:35:17.041435 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 10:35:17.041435 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 27 10:35:17.041435 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Jul 27 10:35:17.046445 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 352/720] 10.222.0.36 () {56 vars in 1102 bytes} [Wed Jul 27 10:35:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 10:35:17.046970 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9851861-b7fe-47fa-b735-0c18658e819a None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:17.047605 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124f6a18-bb96-4484-a578-54f307cd9bfc None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:17.063015 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:17.085919 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:17.158487 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124f6a18-bb96-4484-a578-54f307cd9bfc None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:17.184295 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9851861-b7fe-47fa-b735-0c18658e819a None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:17.197487 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2ef63a-1230-4cda-8855-babe2acb0dfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:17.198417 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2ef63a-1230-4cda-8855-babe2acb0dfc None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:17.198964 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2ef63a-1230-4cda-8855-babe2acb0dfc None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:17.199454 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2ef63a-1230-4cda-8855-babe2acb0dfc None admin] PATH_INFO: `/v3/projects/service` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:17.208694 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2ef63a-1230-4cda-8855-babe2acb0dfc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:17.210169 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2ef63a-1230-4cda-8855-babe2acb0dfc None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:17.228611 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22a4f19-5905-4ec5-aff2-043008a4a17b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:17.229236 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22a4f19-5905-4ec5-aff2-043008a4a17b None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:17.229520 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22a4f19-5905-4ec5-aff2-043008a4a17b None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:17.229800 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22a4f19-5905-4ec5-aff2-043008a4a17b None admin] PATH_INFO: `/v3/roles` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:17.230519 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22a4f19-5905-4ec5-aff2-043008a4a17b None admin] RBAC: Adding query filter params (name=service) {{(pid=86828) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:35:17.230878 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22a4f19-5905-4ec5-aff2-043008a4a17b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:17.231112 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application [None req-5f2ef63a-1230-4cda-8855-babe2acb0dfc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 10:35:17.231112 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:35:17.231112 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:35:17.231112 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:35:17.231112 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:35:17.231112 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:35:17.231112 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:35:17.231112 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:35:17.231112 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:35:17.231112 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:35:17.231112 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:35:17.231112 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:35:17.231112 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 10:35:17.231112 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 10:35:17.231112 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 10:35:17.231953 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 10:35:17.231953 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:35:17.231953 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:35:17.231953 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:35:17.231953 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:35:17.231953 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:35:17.231953 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:35:17.231953 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:35:17.231953 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application with Lock( Jul 27 10:35:17.231953 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:35:17.231953 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:35:17.231953 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:35:17.231953 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:35:17.231953 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:35:17.231953 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:35:17.231953 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:35:17.231953 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:35:17.232832 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 10:35:17.232832 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 10:35:17.232832 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 10:35:17.232832 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 10:35:17.232832 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 10:35:17.232832 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 10:35:17.232832 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 10:35:17.232832 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Jul 27 10:35:17.233253 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22a4f19-5905-4ec5-aff2-043008a4a17b None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:17.235985 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 369/721] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Jul 27 10:35:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 10:35:17.238215 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfbd7ba-ac19-4e75-90b4-6f44a711f12a None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:17.239003 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfbd7ba-ac19-4e75-90b4-6f44a711f12a None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:17.239484 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfbd7ba-ac19-4e75-90b4-6f44a711f12a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:17.245691 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 353/722] 10.222.0.36 () {56 vars in 1103 bytes} [Wed Jul 27 10:35:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 10:35:17.245691 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22a4f19-5905-4ec5-aff2-043008a4a17b None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:17.269664 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-ccfbd7ba-ac19-4e75-90b4-6f44a711f12a None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:35:17.271848 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-ccfbd7ba-ac19-4e75-90b4-6f44a711f12a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:35:17.332543 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:17.349044 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 370/723] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 10:35:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 10:35:17.350423 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfbd7ba-ac19-4e75-90b4-6f44a711f12a None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:17.380259 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:17.390080 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22a4f19-5905-4ec5-aff2-043008a4a17b None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:17.425982 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3658a171-d8d3-4370-9252-e49cb408dcd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:17.426727 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3658a171-d8d3-4370-9252-e49cb408dcd2 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:17.426998 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3658a171-d8d3-4370-9252-e49cb408dcd2 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:17.427272 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3658a171-d8d3-4370-9252-e49cb408dcd2 None admin] PATH_INFO: `/v3/projects` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:17.428078 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3658a171-d8d3-4370-9252-e49cb408dcd2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86828) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:35:17.428931 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3658a171-d8d3-4370-9252-e49cb408dcd2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:17.430367 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3658a171-d8d3-4370-9252-e49cb408dcd2 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:17.443732 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 354/724] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Jul 27 10:35:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:35:17.444902 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3658a171-d8d3-4370-9252-e49cb408dcd2 None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:17.481712 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfbd7ba-ac19-4e75-90b4-6f44a711f12a None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:17.529135 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac0a2cc-59cf-49a9-a8fe-71cea197f8bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:17.530064 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac0a2cc-59cf-49a9-a8fe-71cea197f8bd None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:17.530501 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac0a2cc-59cf-49a9-a8fe-71cea197f8bd None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:17.531220 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac0a2cc-59cf-49a9-a8fe-71cea197f8bd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:17.545300 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac0a2cc-59cf-49a9-a8fe-71cea197f8bd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:17.547247 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac0a2cc-59cf-49a9-a8fe-71cea197f8bd None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:17.555117 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3658a171-d8d3-4370-9252-e49cb408dcd2 None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:17.566348 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application [None req-0ac0a2cc-59cf-49a9-a8fe-71cea197f8bd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 10:35:17.566348 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:35:17.566348 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:35:17.566348 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:35:17.566348 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:35:17.566348 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:35:17.566348 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:35:17.566348 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:35:17.566348 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:35:17.566348 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:35:17.566348 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:35:17.566348 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:35:17.566348 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 10:35:17.566348 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 10:35:17.566348 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 10:35:17.567448 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 10:35:17.567448 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:35:17.567448 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:35:17.567448 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:35:17.567448 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:35:17.567448 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:35:17.567448 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:35:17.567448 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:35:17.567448 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application with Lock( Jul 27 10:35:17.567448 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:35:17.567448 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:35:17.567448 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:35:17.567448 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:35:17.567448 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:35:17.567448 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:35:17.567448 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:35:17.567448 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:35:17.568595 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 10:35:17.568595 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 10:35:17.568595 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 10:35:17.568595 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Jul 27 10:35:17.571035 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 371/725] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 10:35:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 10:35:17.572032 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac0a2cc-59cf-49a9-a8fe-71cea197f8bd None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:17.594523 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:17.602849 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b411b078-da30-427c-ad72-360cf01dc64a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:17.603868 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b411b078-da30-427c-ad72-360cf01dc64a None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:17.604308 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b411b078-da30-427c-ad72-360cf01dc64a None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:17.604753 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b411b078-da30-427c-ad72-360cf01dc64a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:17.617254 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b411b078-da30-427c-ad72-360cf01dc64a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 10:35:17.622771 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application [None req-b411b078-da30-427c-ad72-360cf01dc64a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 10:35:17.622771 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:35:17.622771 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:35:17.622771 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:35:17.622771 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:35:17.622771 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:35:17.622771 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:35:17.622771 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:35:17.622771 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:35:17.622771 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:35:17.622771 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:35:17.622771 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:35:17.622771 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 10:35:17.622771 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 10:35:17.622771 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 10:35:17.622771 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 10:35:17.623871 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 10:35:17.623871 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 10:35:17.623871 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 10:35:17.623871 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 10:35:17.623871 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:35:17.623871 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:35:17.623871 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:35:17.623871 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:35:17.623871 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:35:17.623871 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:35:17.623871 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:35:17.623871 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application with Lock( Jul 27 10:35:17.623871 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:35:17.623871 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:35:17.623871 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:35:17.623871 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:35:17.624907 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:35:17.624907 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:35:17.624907 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:35:17.624907 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:35:17.624907 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 10:35:17.624907 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 10:35:17.624907 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 10:35:17.624907 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 10:35:17.624907 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 10:35:17.624907 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 10:35:17.624907 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 10:35:17.624907 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Jul 27 10:35:17.631624 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 355/726] 10.222.0.36 () {56 vars in 1096 bytes} [Wed Jul 27 10:35:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 10:35:17.631624 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b411b078-da30-427c-ad72-360cf01dc64a None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:17.671436 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:17.699935 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac0a2cc-59cf-49a9-a8fe-71cea197f8bd None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:17.736873 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc20b15-f5f5-4122-af68-62bb7d7975bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:17.737635 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc20b15-f5f5-4122-af68-62bb7d7975bf None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:17.738013 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc20b15-f5f5-4122-af68-62bb7d7975bf None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:17.738404 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc20b15-f5f5-4122-af68-62bb7d7975bf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:17.739551 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc20b15-f5f5-4122-af68-62bb7d7975bf None admin] RBAC: Adding query filter params (role.id=21156560eb6b436bb567a40b4976f3f4, scope.project.id=5f296d8e3b8f4001909addc49d8dac01, user.id=e059d8b6c28c4fcba925d05d7c98e4ac) {{(pid=86827) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:35:17.739857 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc20b15-f5f5-4122-af68-62bb7d7975bf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:17.741952 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc20b15-f5f5-4122-af68-62bb7d7975bf None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:17.748604 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 372/727] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Jul 27 10:35:17 2022] GET /identity/v3/role_assignments?user.id=e059d8b6c28c4fcba925d05d7c98e4ac&scope.project.id=5f296d8e3b8f4001909addc49d8dac01&role.id=21156560eb6b436bb567a40b4976f3f4 => generated 259 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:35:17.749842 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc20b15-f5f5-4122-af68-62bb7d7975bf None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:17.754746 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b411b078-da30-427c-ad72-360cf01dc64a None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:17.797045 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f08371-0388-4f2d-9fe0-42088545b6ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:17.797875 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f08371-0388-4f2d-9fe0-42088545b6ce None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:17.798253 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f08371-0388-4f2d-9fe0-42088545b6ce None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:17.798656 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f08371-0388-4f2d-9fe0-42088545b6ce None admin] PATH_INFO: `/v3/domains` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:17.799560 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f08371-0388-4f2d-9fe0-42088545b6ce None admin] RBAC: Adding query filter params (name=Default) {{(pid=86828) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:35:17.799980 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f08371-0388-4f2d-9fe0-42088545b6ce None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:17.801582 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f08371-0388-4f2d-9fe0-42088545b6ce None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:17.820858 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 356/728] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 10:35:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:35:17.822632 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f08371-0388-4f2d-9fe0-42088545b6ce None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:17.880049 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc20b15-f5f5-4122-af68-62bb7d7975bf None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:17.880396 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:17.928161 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035572db-98a9-45a8-b48c-b7c77a81ab78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:17.929114 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035572db-98a9-45a8-b48c-b7c77a81ab78 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:17.931307 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035572db-98a9-45a8-b48c-b7c77a81ab78 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:17.931796 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035572db-98a9-45a8-b48c-b7c77a81ab78 None admin] PATH_INFO: `/v3/roles` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:17.932711 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035572db-98a9-45a8-b48c-b7c77a81ab78 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86827) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:35:17.938316 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035572db-98a9-45a8-b48c-b7c77a81ab78 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:17.939977 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035572db-98a9-45a8-b48c-b7c77a81ab78 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:17.959415 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 373/729] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jul 27 10:35:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:35:17.963674 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035572db-98a9-45a8-b48c-b7c77a81ab78 None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:17.969742 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:17.987320 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f08371-0388-4f2d-9fe0-42088545b6ce None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:18.025186 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07af84e4-5607-4152-9900-498411e5665d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:18.026338 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07af84e4-5607-4152-9900-498411e5665d None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:18.026863 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07af84e4-5607-4152-9900-498411e5665d None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:18.027399 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07af84e4-5607-4152-9900-498411e5665d None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:18.036227 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07af84e4-5607-4152-9900-498411e5665d None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:18.038433 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07af84e4-5607-4152-9900-498411e5665d None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:18.052839 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application [None req-07af84e4-5607-4152-9900-498411e5665d None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 27 10:35:18.052839 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:35:18.052839 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:35:18.052839 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:35:18.052839 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:35:18.052839 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:35:18.052839 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:35:18.052839 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:35:18.052839 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:35:18.052839 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:35:18.052839 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:35:18.052839 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:35:18.052839 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 10:35:18.052839 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 10:35:18.052839 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 10:35:18.052839 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 10:35:18.053885 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:35:18.053885 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:35:18.053885 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 10:35:18.053885 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 10:35:18.053885 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 10:35:18.053885 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 10:35:18.053885 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:35:18.053885 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:35:18.053885 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:35:18.053885 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:35:18.053885 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:35:18.053885 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application with Lock( Jul 27 10:35:18.053885 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:35:18.053885 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:35:18.053885 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:35:18.053885 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:35:18.053885 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:35:18.053885 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:35:18.054927 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:35:18.054927 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:35:18.054927 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 10:35:18.054927 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 10:35:18.054927 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 10:35:18.054927 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 10:35:18.054927 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 10:35:18.054927 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 10:35:18.054927 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 27 10:35:18.054927 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Jul 27 10:35:18.062068 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 357/730] 10.222.0.36 () {56 vars in 1117 bytes} [Wed Jul 27 10:35:17 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 10:35:18.063747 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85d7e3f-4545-4d67-b846-d4e7de478fb3 None None] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:18.064308 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85d7e3f-4545-4d67-b846-d4e7de478fb3 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:18.064911 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85d7e3f-4545-4d67-b846-d4e7de478fb3 None None] PATH_INFO: `/` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:18.066397 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 358/731] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 10:35:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 10:35:18.071459 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85d7e3f-4545-4d67-b846-d4e7de478fb3 None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:18.073805 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035572db-98a9-45a8-b48c-b7c77a81ab78 None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:18.109377 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc415fb6-9615-4dfa-8884-45ec88fcf70c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:18.110041 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc415fb6-9615-4dfa-8884-45ec88fcf70c None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:18.110303 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc415fb6-9615-4dfa-8884-45ec88fcf70c None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:18.110579 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc415fb6-9615-4dfa-8884-45ec88fcf70c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:18.122400 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc415fb6-9615-4dfa-8884-45ec88fcf70c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:18.123867 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc415fb6-9615-4dfa-8884-45ec88fcf70c None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:18.138527 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application [None req-cc415fb6-9615-4dfa-8884-45ec88fcf70c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 10:35:18.138527 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:35:18.138527 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:35:18.138527 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:35:18.138527 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:35:18.138527 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:35:18.138527 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:35:18.138527 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:35:18.138527 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:35:18.138527 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:35:18.138527 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:35:18.138527 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:35:18.138527 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 10:35:18.138527 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 10:35:18.138527 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 10:35:18.139456 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 10:35:18.139456 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:35:18.139456 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:35:18.139456 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:35:18.139456 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:35:18.139456 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:35:18.139456 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:35:18.139456 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:35:18.139456 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application with Lock( Jul 27 10:35:18.139456 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:35:18.139456 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:35:18.139456 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:35:18.139456 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:35:18.139456 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:35:18.139456 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:35:18.139456 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:35:18.139456 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:35:18.145051 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 10:35:18.145051 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 10:35:18.145051 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 10:35:18.145051 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Jul 27 10:35:18.145051 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 374/732] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 10:35:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 10:35:18.145051 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800915c8-dcf3-43e4-9fb9-02b544c0ccbd None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:18.145051 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800915c8-dcf3-43e4-9fb9-02b544c0ccbd None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:18.145051 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800915c8-dcf3-43e4-9fb9-02b544c0ccbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:18.159884 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-800915c8-dcf3-43e4-9fb9-02b544c0ccbd None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:35:18.161668 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-800915c8-dcf3-43e4-9fb9-02b544c0ccbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:35:18.195931 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85d7e3f-4545-4d67-b846-d4e7de478fb3 None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:18.207686 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:18.212270 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:18.234644 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 375/733] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 10:35:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 10:35:18.234644 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800915c8-dcf3-43e4-9fb9-02b544c0ccbd None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:18.252324 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da385cf8-87b4-48d9-bbf6-f92f08b666bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:18.253275 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da385cf8-87b4-48d9-bbf6-f92f08b666bd None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:18.253798 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da385cf8-87b4-48d9-bbf6-f92f08b666bd None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:18.254279 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da385cf8-87b4-48d9-bbf6-f92f08b666bd None admin] PATH_INFO: `/v3/users` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:18.255428 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da385cf8-87b4-48d9-bbf6-f92f08b666bd None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=86828) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:35:18.255996 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da385cf8-87b4-48d9-bbf6-f92f08b666bd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:18.258166 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da385cf8-87b4-48d9-bbf6-f92f08b666bd None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:18.293909 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 359/734] 10.222.0.36 () {56 vars in 1149 bytes} [Wed Jul 27 10:35:18 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:35:18.293909 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da385cf8-87b4-48d9-bbf6-f92f08b666bd None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:18.368171 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800915c8-dcf3-43e4-9fb9-02b544c0ccbd None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:18.400930 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da385cf8-87b4-48d9-bbf6-f92f08b666bd None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:18.418976 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af93ffd9-3842-46ad-b7de-7b65b5a2525e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:18.419702 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af93ffd9-3842-46ad-b7de-7b65b5a2525e None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:18.420420 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af93ffd9-3842-46ad-b7de-7b65b5a2525e None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:18.420770 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af93ffd9-3842-46ad-b7de-7b65b5a2525e None admin] PATH_INFO: `/v3/domains` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:18.421546 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af93ffd9-3842-46ad-b7de-7b65b5a2525e None admin] RBAC: Adding query filter params (name=Default) {{(pid=86827) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:35:18.421944 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af93ffd9-3842-46ad-b7de-7b65b5a2525e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:18.423471 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af93ffd9-3842-46ad-b7de-7b65b5a2525e None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:18.444592 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d1e726-f333-48c4-9b60-88e69a3dfc95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:18.444592 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d1e726-f333-48c4-9b60-88e69a3dfc95 None admin] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:18.444592 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d1e726-f333-48c4-9b60-88e69a3dfc95 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:18.444592 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d1e726-f333-48c4-9b60-88e69a3dfc95 None admin] PATH_INFO: `/v3/services` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:18.445079 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d1e726-f333-48c4-9b60-88e69a3dfc95 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:18.445079 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d1e726-f333-48c4-9b60-88e69a3dfc95 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:18.445544 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 376/735] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 10:35:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:35:18.446680 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af93ffd9-3842-46ad-b7de-7b65b5a2525e None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:18.471595 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 360/736] 10.222.0.36 () {60 vars in 1141 bytes} [Wed Jul 27 10:35:18 2022] POST /identity/v3/services => generated 251 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 10:35:18.471595 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d1e726-f333-48c4-9b60-88e69a3dfc95 None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:18.504232 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:18.529620 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:18.629341 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d1e726-f333-48c4-9b60-88e69a3dfc95 None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:18.668522 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af93ffd9-3842-46ad-b7de-7b65b5a2525e None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:18.674122 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df5c990-2e68-47f1-828e-6cd7cd6a58f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:18.674811 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df5c990-2e68-47f1-828e-6cd7cd6a58f0 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:18.675097 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df5c990-2e68-47f1-828e-6cd7cd6a58f0 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:18.675375 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df5c990-2e68-47f1-828e-6cd7cd6a58f0 None admin] PATH_INFO: `/v3/users/swift` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:18.683762 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df5c990-2e68-47f1-828e-6cd7cd6a58f0 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:18.685793 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df5c990-2e68-47f1-828e-6cd7cd6a58f0 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:18.700978 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application [None req-2df5c990-2e68-47f1-828e-6cd7cd6a58f0 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 27 10:35:18.700978 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:35:18.700978 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:35:18.700978 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:35:18.700978 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:35:18.700978 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:35:18.700978 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:35:18.700978 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:35:18.700978 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:35:18.700978 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:35:18.700978 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:35:18.700978 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:35:18.700978 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 10:35:18.700978 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 10:35:18.700978 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 10:35:18.700978 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 10:35:18.701987 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:35:18.701987 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:35:18.701987 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 10:35:18.701987 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 10:35:18.701987 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 10:35:18.701987 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 10:35:18.701987 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:35:18.701987 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:35:18.701987 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:35:18.701987 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:35:18.701987 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:35:18.701987 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application with Lock( Jul 27 10:35:18.701987 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:35:18.701987 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:35:18.701987 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:35:18.701987 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:35:18.701987 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:35:18.701987 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:35:18.702963 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:35:18.702963 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:35:18.702963 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 10:35:18.702963 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 10:35:18.702963 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 10:35:18.702963 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 10:35:18.702963 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 10:35:18.702963 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 10:35:18.702963 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 27 10:35:18.702963 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Jul 27 10:35:18.707270 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 361/737] 10.222.0.36 () {56 vars in 1096 bytes} [Wed Jul 27 10:35:18 2022] GET /identity/v3/users/swift => generated 83 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 10:35:18.708659 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df5c990-2e68-47f1-828e-6cd7cd6a58f0 None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:18.754869 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a24ec6a-d7aa-4d5a-b013-f938791a101a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:18.756303 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a24ec6a-d7aa-4d5a-b013-f938791a101a None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:18.756796 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a24ec6a-d7aa-4d5a-b013-f938791a101a None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:18.757292 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a24ec6a-d7aa-4d5a-b013-f938791a101a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:18.815053 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:18.838649 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a24ec6a-d7aa-4d5a-b013-f938791a101a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:18.838844 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df5c990-2e68-47f1-828e-6cd7cd6a58f0 None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:18.840573 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a24ec6a-d7aa-4d5a-b013-f938791a101a None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:18.883643 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40807bb8-699a-44ad-91d5-6fe7f09d1f68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:18.885213 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40807bb8-699a-44ad-91d5-6fe7f09d1f68 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:18.885414 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40807bb8-699a-44ad-91d5-6fe7f09d1f68 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:18.885649 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40807bb8-699a-44ad-91d5-6fe7f09d1f68 None admin] PATH_INFO: `/v3/users` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:18.886458 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40807bb8-699a-44ad-91d5-6fe7f09d1f68 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=86828) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:35:18.886799 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40807bb8-699a-44ad-91d5-6fe7f09d1f68 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:18.888708 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40807bb8-699a-44ad-91d5-6fe7f09d1f68 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:18.894004 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:18.916119 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 362/738] 10.222.0.36 () {56 vars in 1135 bytes} [Wed Jul 27 10:35:18 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:35:18.922584 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40807bb8-699a-44ad-91d5-6fe7f09d1f68 None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:18.985302 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 377/739] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jul 27 10:35:18 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 537 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 10:35:18.987042 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae556e2-1516-4a2e-ba45-9bba67bae9f5 None None] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:18.988042 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae556e2-1516-4a2e-ba45-9bba67bae9f5 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:18.988527 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae556e2-1516-4a2e-ba45-9bba67bae9f5 None None] PATH_INFO: `/` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:18.991765 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 378/740] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 10:35:18 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 10:35:18.992980 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae556e2-1516-4a2e-ba45-9bba67bae9f5 None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:19.056395 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40807bb8-699a-44ad-91d5-6fe7f09d1f68 None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:19.091694 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d323be-d761-48de-9312-eb0191288d96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:19.092469 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d323be-d761-48de-9312-eb0191288d96 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:19.092854 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d323be-d761-48de-9312-eb0191288d96 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:19.093234 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d323be-d761-48de-9312-eb0191288d96 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:19.106869 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:19.134772 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae556e2-1516-4a2e-ba45-9bba67bae9f5 None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:19.166420 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d323be-d761-48de-9312-eb0191288d96 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:19.167903 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d323be-d761-48de-9312-eb0191288d96 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:19.181770 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89929fc2-5995-48a2-9a50-47852ca7d873 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:19.182995 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89929fc2-5995-48a2-9a50-47852ca7d873 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:19.183984 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89929fc2-5995-48a2-9a50-47852ca7d873 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:19.184612 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89929fc2-5995-48a2-9a50-47852ca7d873 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:19.199997 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89929fc2-5995-48a2-9a50-47852ca7d873 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:19.202393 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89929fc2-5995-48a2-9a50-47852ca7d873 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:19.206504 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:19.226567 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application [None req-89929fc2-5995-48a2-9a50-47852ca7d873 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 10:35:19.226567 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:35:19.226567 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:35:19.226567 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:35:19.226567 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:35:19.226567 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:35:19.226567 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:35:19.226567 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:35:19.226567 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:35:19.226567 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:35:19.226567 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:35:19.226567 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:35:19.226567 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 10:35:19.226567 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 10:35:19.226567 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 10:35:19.227542 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 10:35:19.227542 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:35:19.227542 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:35:19.227542 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:35:19.227542 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:35:19.227542 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:35:19.227542 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:35:19.227542 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:35:19.227542 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application with Lock( Jul 27 10:35:19.227542 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:35:19.227542 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:35:19.227542 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:35:19.227542 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:35:19.227542 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:35:19.227542 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:35:19.227542 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:35:19.227542 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:35:19.228486 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 10:35:19.228486 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 10:35:19.228486 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 10:35:19.228486 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Jul 27 10:35:19.231385 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 379/741] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 10:35:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 10:35:19.233617 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9b30f0-ee21-4841-9af3-f7b1d0933f59 None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:19.234233 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9b30f0-ee21-4841-9af3-f7b1d0933f59 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:19.234763 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9b30f0-ee21-4841-9af3-f7b1d0933f59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:19.259515 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-7e9b30f0-ee21-4841-9af3-f7b1d0933f59 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:35:19.262483 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-7e9b30f0-ee21-4841-9af3-f7b1d0933f59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:35:19.283262 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 363/742] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jul 27 10:35:18 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 10:35:19.284164 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d323be-d761-48de-9312-eb0191288d96 None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:19.336122 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:19.347996 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 380/743] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 10:35:19 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 10:35:19.348999 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9b30f0-ee21-4841-9af3-f7b1d0933f59 None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:19.401681 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d323be-d761-48de-9312-eb0191288d96 None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:19.442697 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5baefc6-f0be-4649-a740-e969ff698812 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:19.443720 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5baefc6-f0be-4649-a740-e969ff698812 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:19.444865 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5baefc6-f0be-4649-a740-e969ff698812 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:19.445408 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5baefc6-f0be-4649-a740-e969ff698812 None admin] PATH_INFO: `/v3/domains` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:19.446338 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5baefc6-f0be-4649-a740-e969ff698812 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86828) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:35:19.446923 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5baefc6-f0be-4649-a740-e969ff698812 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:19.448802 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5baefc6-f0be-4649-a740-e969ff698812 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:19.457283 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9b30f0-ee21-4841-9af3-f7b1d0933f59 None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:19.462462 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 364/744] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 10:35:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:35:19.463605 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5baefc6-f0be-4649-a740-e969ff698812 None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:19.492800 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:19.504564 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896e0480-bfbc-4ed2-bc84-7729d3216bc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:19.505234 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896e0480-bfbc-4ed2-bc84-7729d3216bc0 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:19.505513 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896e0480-bfbc-4ed2-bc84-7729d3216bc0 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:19.505786 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896e0480-bfbc-4ed2-bc84-7729d3216bc0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:19.519221 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896e0480-bfbc-4ed2-bc84-7729d3216bc0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:19.520805 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896e0480-bfbc-4ed2-bc84-7729d3216bc0 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:19.538529 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application [None req-896e0480-bfbc-4ed2-bc84-7729d3216bc0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 10:35:19.538529 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:35:19.538529 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:35:19.538529 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:35:19.538529 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:35:19.538529 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:35:19.538529 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:35:19.538529 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:35:19.538529 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:35:19.538529 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:35:19.538529 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:35:19.538529 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:35:19.538529 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 10:35:19.538529 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 10:35:19.538529 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 10:35:19.539591 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 10:35:19.539591 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:35:19.539591 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:35:19.539591 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:35:19.539591 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:35:19.539591 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:35:19.539591 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:35:19.539591 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:35:19.539591 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application with Lock( Jul 27 10:35:19.539591 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:35:19.539591 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:35:19.539591 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:35:19.539591 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:35:19.539591 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:35:19.539591 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:35:19.539591 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:35:19.539591 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:35:19.540532 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 10:35:19.540532 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 10:35:19.540532 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 10:35:19.540532 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Jul 27 10:35:19.544596 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 381/745] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 10:35:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 10:35:19.545586 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896e0480-bfbc-4ed2-bc84-7729d3216bc0 None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:19.665216 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896e0480-bfbc-4ed2-bc84-7729d3216bc0 None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:19.715779 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da0b891-8483-43a5-beff-a2fa63eb13aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:19.716410 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da0b891-8483-43a5-beff-a2fa63eb13aa None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:19.716720 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da0b891-8483-43a5-beff-a2fa63eb13aa None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:19.717007 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da0b891-8483-43a5-beff-a2fa63eb13aa None admin] PATH_INFO: `/v3/projects/service` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:19.725537 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da0b891-8483-43a5-beff-a2fa63eb13aa None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:19.726281 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5baefc6-f0be-4649-a740-e969ff698812 None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:19.727082 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da0b891-8483-43a5-beff-a2fa63eb13aa None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:19.728860 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:19.743665 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application [None req-2da0b891-8483-43a5-beff-a2fa63eb13aa None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 10:35:19.743665 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:35:19.743665 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:35:19.743665 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:35:19.743665 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:35:19.743665 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:35:19.743665 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:35:19.743665 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:35:19.743665 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:35:19.743665 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:35:19.743665 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:35:19.743665 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:35:19.743665 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 10:35:19.743665 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 10:35:19.743665 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 10:35:19.744833 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 10:35:19.744833 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:35:19.744833 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:35:19.744833 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:35:19.744833 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:35:19.744833 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:35:19.744833 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:35:19.744833 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:35:19.744833 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application with Lock( Jul 27 10:35:19.744833 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:35:19.744833 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:35:19.744833 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:35:19.744833 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:35:19.744833 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:35:19.744833 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:35:19.744833 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:35:19.744833 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:35:19.745933 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 10:35:19.745933 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 10:35:19.745933 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 10:35:19.745933 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 10:35:19.745933 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 10:35:19.745933 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 10:35:19.745933 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 10:35:19.745933 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Jul 27 10:35:19.750046 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 382/746] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Jul 27 10:35:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 10:35:19.751063 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da0b891-8483-43a5-beff-a2fa63eb13aa None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:19.774350 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c4c738-c9dc-47df-a545-7a2994b53138 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:19.775495 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c4c738-c9dc-47df-a545-7a2994b53138 None admin] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:19.776101 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c4c738-c9dc-47df-a545-7a2994b53138 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:19.776670 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c4c738-c9dc-47df-a545-7a2994b53138 None admin] PATH_INFO: `/v3/services` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:19.777728 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c4c738-c9dc-47df-a545-7a2994b53138 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:19.779601 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c4c738-c9dc-47df-a545-7a2994b53138 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:19.800405 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 365/747] 10.222.0.36 () {60 vars in 1141 bytes} [Wed Jul 27 10:35:19 2022] POST /identity/v3/services => generated 240 bytes in 337 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 10:35:19.804927 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:19.806148 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c4c738-c9dc-47df-a545-7a2994b53138 None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:19.891083 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da0b891-8483-43a5-beff-a2fa63eb13aa None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:19.948516 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc01b8de-b15b-462f-bc99-9f2e66fa2386 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:19.949504 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc01b8de-b15b-462f-bc99-9f2e66fa2386 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:19.949967 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc01b8de-b15b-462f-bc99-9f2e66fa2386 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:19.950420 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc01b8de-b15b-462f-bc99-9f2e66fa2386 None admin] PATH_INFO: `/v3/domains` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:19.951328 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc01b8de-b15b-462f-bc99-9f2e66fa2386 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86827) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:35:19.952138 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc01b8de-b15b-462f-bc99-9f2e66fa2386 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:19.954105 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc01b8de-b15b-462f-bc99-9f2e66fa2386 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:19.963125 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c4c738-c9dc-47df-a545-7a2994b53138 None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:19.972812 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 383/748] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 10:35:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:35:19.974158 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39620ad0-eb0a-47d9-832e-6e20299bf0fd None None] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:19.974458 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39620ad0-eb0a-47d9-832e-6e20299bf0fd None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:19.974762 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39620ad0-eb0a-47d9-832e-6e20299bf0fd None None] PATH_INFO: `/` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:19.977262 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 384/749] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 10:35:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 10:35:19.978238 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39620ad0-eb0a-47d9-832e-6e20299bf0fd None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:20.008722 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90edaaa-0d6a-4d36-a3e2-90c2721bf719 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:20.009607 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90edaaa-0d6a-4d36-a3e2-90c2721bf719 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:20.009893 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90edaaa-0d6a-4d36-a3e2-90c2721bf719 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:20.010168 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90edaaa-0d6a-4d36-a3e2-90c2721bf719 None admin] PATH_INFO: `/v3/projects` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:20.011184 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90edaaa-0d6a-4d36-a3e2-90c2721bf719 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86828) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:35:20.012116 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90edaaa-0d6a-4d36-a3e2-90c2721bf719 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:20.014422 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90edaaa-0d6a-4d36-a3e2-90c2721bf719 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:20.034906 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 366/750] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Jul 27 10:35:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:35:20.036984 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb38c68b-5f3c-4a1b-874e-75499461cf1d None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:20.038184 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb38c68b-5f3c-4a1b-874e-75499461cf1d None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:20.038711 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb38c68b-5f3c-4a1b-874e-75499461cf1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:20.063454 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-cb38c68b-5f3c-4a1b-874e-75499461cf1d None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:35:20.064670 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-cb38c68b-5f3c-4a1b-874e-75499461cf1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:35:20.090859 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:20.106386 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:20.116587 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39620ad0-eb0a-47d9-832e-6e20299bf0fd None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:20.155365 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5b3498-ed5c-4319-bf03-8e6c94eb95c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:20.156383 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5b3498-ed5c-4319-bf03-8e6c94eb95c5 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:20.156822 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5b3498-ed5c-4319-bf03-8e6c94eb95c5 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:20.157252 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5b3498-ed5c-4319-bf03-8e6c94eb95c5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:20.157429 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 367/751] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 10:35:20 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 10:35:20.158219 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb38c68b-5f3c-4a1b-874e-75499461cf1d None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:20.165945 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5b3498-ed5c-4319-bf03-8e6c94eb95c5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:20.167631 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5b3498-ed5c-4319-bf03-8e6c94eb95c5 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:20.179568 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application [None req-dc5b3498-ed5c-4319-bf03-8e6c94eb95c5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 10:35:20.179568 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:35:20.179568 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:35:20.179568 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:35:20.179568 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:35:20.179568 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:35:20.179568 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:35:20.179568 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:35:20.179568 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:35:20.179568 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:35:20.179568 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:35:20.179568 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:35:20.179568 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 10:35:20.179568 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 10:35:20.179568 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 10:35:20.180659 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 10:35:20.180659 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:35:20.180659 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:35:20.180659 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:35:20.180659 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:35:20.180659 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:35:20.180659 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:35:20.180659 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:35:20.180659 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application with Lock( Jul 27 10:35:20.180659 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:35:20.180659 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:35:20.180659 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:35:20.180659 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:35:20.180659 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:35:20.180659 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:35:20.180659 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:35:20.180659 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:35:20.181665 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 10:35:20.181665 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 10:35:20.181665 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 10:35:20.181665 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 10:35:20.181665 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 10:35:20.181665 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 10:35:20.181665 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 10:35:20.181665 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Jul 27 10:35:20.187981 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 385/752] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Jul 27 10:35:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 10:35:20.189316 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5b3498-ed5c-4319-bf03-8e6c94eb95c5 None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:20.264964 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb38c68b-5f3c-4a1b-874e-75499461cf1d None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:20.296296 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5b3498-ed5c-4319-bf03-8e6c94eb95c5 None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:20.302287 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022ce110-f8d9-4b08-9125-fa874596a2ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:20.302947 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022ce110-f8d9-4b08-9125-fa874596a2ba None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:20.303263 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022ce110-f8d9-4b08-9125-fa874596a2ba None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:20.303540 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022ce110-f8d9-4b08-9125-fa874596a2ba None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:20.304408 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022ce110-f8d9-4b08-9125-fa874596a2ba None admin] RBAC: Adding query filter params (role.id=9e0038c6bbd34556af3f84f9e2a7c2c5, scope.project.id=5f296d8e3b8f4001909addc49d8dac01, user.id=3d9fed24b4ba44b0b5961a603c4362c4) {{(pid=86828) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:35:20.304719 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022ce110-f8d9-4b08-9125-fa874596a2ba None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:20.306258 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022ce110-f8d9-4b08-9125-fa874596a2ba None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:20.312393 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 368/753] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Jul 27 10:35:20 2022] GET /identity/v3/role_assignments?user.id=3d9fed24b4ba44b0b5961a603c4362c4&scope.project.id=5f296d8e3b8f4001909addc49d8dac01&role.id=9e0038c6bbd34556af3f84f9e2a7c2c5 => generated 259 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:35:20.313502 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022ce110-f8d9-4b08-9125-fa874596a2ba None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:20.332880 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:20.336960 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c4d5dc-9326-489d-ab34-620166ba5723 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:20.337825 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c4d5dc-9326-489d-ab34-620166ba5723 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:20.338262 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c4d5dc-9326-489d-ab34-620166ba5723 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:20.338740 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c4d5dc-9326-489d-ab34-620166ba5723 None admin] PATH_INFO: `/v3/projects` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:20.339680 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c4d5dc-9326-489d-ab34-620166ba5723 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86827) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:35:20.340176 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c4d5dc-9326-489d-ab34-620166ba5723 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:20.341813 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c4d5dc-9326-489d-ab34-620166ba5723 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:20.364843 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 386/754] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Jul 27 10:35:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 10:35:20.367110 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c4d5dc-9326-489d-ab34-620166ba5723 None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:20.385034 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:20.442283 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022ce110-f8d9-4b08-9125-fa874596a2ba None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:20.480486 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e668d1f-6038-4ab4-b446-2718dcd72f28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:20.481145 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e668d1f-6038-4ab4-b446-2718dcd72f28 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:20.481431 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e668d1f-6038-4ab4-b446-2718dcd72f28 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:20.481707 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e668d1f-6038-4ab4-b446-2718dcd72f28 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:20.483653 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c4d5dc-9326-489d-ab34-620166ba5723 None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:20.493795 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e668d1f-6038-4ab4-b446-2718dcd72f28 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 10:35:20.500144 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application [None req-4e668d1f-6038-4ab4-b446-2718dcd72f28 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 10:35:20.500144 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:35:20.500144 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:35:20.500144 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:35:20.500144 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:35:20.500144 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:35:20.500144 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:35:20.500144 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:35:20.500144 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:35:20.500144 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:35:20.500144 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:35:20.500144 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:35:20.500144 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 10:35:20.500144 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 10:35:20.500144 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 10:35:20.500144 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 10:35:20.501200 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 10:35:20.501200 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 10:35:20.501200 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 10:35:20.501200 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 10:35:20.501200 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:35:20.501200 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:35:20.501200 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:35:20.501200 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:35:20.501200 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:35:20.501200 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:35:20.501200 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:35:20.501200 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application with Lock( Jul 27 10:35:20.501200 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:35:20.501200 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:35:20.501200 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:35:20.501200 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:35:20.502264 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:35:20.502264 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:35:20.502264 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:35:20.502264 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:35:20.502264 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 10:35:20.502264 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 10:35:20.502264 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 10:35:20.502264 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 10:35:20.502264 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 10:35:20.502264 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 10:35:20.502264 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 10:35:20.502264 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Jul 27 10:35:20.502264 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 369/755] 10.222.0.36 () {56 vars in 1096 bytes} [Wed Jul 27 10:35:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 10:35:20.508643 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e668d1f-6038-4ab4-b446-2718dcd72f28 None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:20.540248 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62927ff-b907-4dfb-86aa-4ca9a604159c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:20.540802 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62927ff-b907-4dfb-86aa-4ca9a604159c None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:20.541081 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62927ff-b907-4dfb-86aa-4ca9a604159c None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:20.541358 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62927ff-b907-4dfb-86aa-4ca9a604159c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:20.542164 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62927ff-b907-4dfb-86aa-4ca9a604159c None admin] RBAC: Adding query filter params (role.id=21156560eb6b436bb567a40b4976f3f4, scope.project.id=5f296d8e3b8f4001909addc49d8dac01, user.id=89ac506eefcd4e42a84b44fa15fb3975) {{(pid=86827) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:35:20.542475 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62927ff-b907-4dfb-86aa-4ca9a604159c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:20.543954 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62927ff-b907-4dfb-86aa-4ca9a604159c None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:20.556334 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 387/756] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Jul 27 10:35:20 2022] GET /identity/v3/role_assignments?user.id=89ac506eefcd4e42a84b44fa15fb3975&scope.project.id=5f296d8e3b8f4001909addc49d8dac01&role.id=21156560eb6b436bb567a40b4976f3f4 => generated 259 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 10:35:20.616916 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:20.655677 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e668d1f-6038-4ab4-b446-2718dcd72f28 None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:20.708883 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c647336-588e-41f5-9475-84adf0a58368 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:20.710107 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c647336-588e-41f5-9475-84adf0a58368 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:20.710395 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c647336-588e-41f5-9475-84adf0a58368 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:20.710697 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c647336-588e-41f5-9475-84adf0a58368 None admin] PATH_INFO: `/v3/roles` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:20.711459 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c647336-588e-41f5-9475-84adf0a58368 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86828) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:35:20.711768 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c647336-588e-41f5-9475-84adf0a58368 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:20.713186 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c647336-588e-41f5-9475-84adf0a58368 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:20.727356 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 370/757] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jul 27 10:35:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 10:35:20.732739 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62927ff-b907-4dfb-86aa-4ca9a604159c None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:20.814442 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:20.839618 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62927ff-b907-4dfb-86aa-4ca9a604159c None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:20.883538 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393ddbd8-93ab-4d45-bdee-8dd49e88ad5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:20.884434 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393ddbd8-93ab-4d45-bdee-8dd49e88ad5a None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:20.884716 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393ddbd8-93ab-4d45-bdee-8dd49e88ad5a None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:20.885074 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393ddbd8-93ab-4d45-bdee-8dd49e88ad5a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:20.895906 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393ddbd8-93ab-4d45-bdee-8dd49e88ad5a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:20.897391 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393ddbd8-93ab-4d45-bdee-8dd49e88ad5a None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:20.911921 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application [None req-393ddbd8-93ab-4d45-bdee-8dd49e88ad5a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 10:35:20.911921 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:35:20.911921 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:35:20.911921 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:35:20.911921 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:35:20.911921 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:35:20.911921 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:35:20.911921 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:35:20.911921 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:35:20.911921 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:35:20.911921 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:35:20.911921 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:35:20.911921 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 10:35:20.911921 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 10:35:20.911921 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 10:35:20.912857 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 10:35:20.912857 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:35:20.912857 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:35:20.912857 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:35:20.912857 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:35:20.912857 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:35:20.912857 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:35:20.912857 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:35:20.912857 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application with Lock( Jul 27 10:35:20.912857 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:35:20.912857 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:35:20.912857 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:35:20.912857 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:35:20.912857 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:35:20.912857 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:35:20.912857 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:35:20.912857 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:35:20.914834 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 10:35:20.914834 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 10:35:20.914834 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 10:35:20.914834 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Jul 27 10:35:20.914834 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 388/758] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 10:35:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 10:35:20.919040 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c647336-588e-41f5-9475-84adf0a58368 None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:21.090656 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c647336-588e-41f5-9475-84adf0a58368 None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:21.140701 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d1a711-486f-48e1-acc7-49dcf4d4e757 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:21.141328 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d1a711-486f-48e1-acc7-49dcf4d4e757 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:21.141608 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d1a711-486f-48e1-acc7-49dcf4d4e757 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:21.141880 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d1a711-486f-48e1-acc7-49dcf4d4e757 None admin] PATH_INFO: `/v3/domains` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:21.142579 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d1a711-486f-48e1-acc7-49dcf4d4e757 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86828) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:35:21.142906 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d1a711-486f-48e1-acc7-49dcf4d4e757 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:21.144272 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d1a711-486f-48e1-acc7-49dcf4d4e757 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:21.156582 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:21.164371 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 371/759] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 10:35:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 10:35:21.169603 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393ddbd8-93ab-4d45-bdee-8dd49e88ad5a None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:21.206559 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd156f58-35b7-4c25-825d-f328d1707fd3 None None] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:21.206927 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd156f58-35b7-4c25-825d-f328d1707fd3 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:21.207151 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd156f58-35b7-4c25-825d-f328d1707fd3 None None] PATH_INFO: `/` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:21.208901 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 372/760] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 10:35:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 10:35:21.213913 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6143c097-8354-43cf-ba9b-3df34d19273f None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:21.214189 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6143c097-8354-43cf-ba9b-3df34d19273f None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:21.214483 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6143c097-8354-43cf-ba9b-3df34d19273f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:21.234773 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-6143c097-8354-43cf-ba9b-3df34d19273f None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:35:21.236409 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-6143c097-8354-43cf-ba9b-3df34d19273f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:35:21.314626 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393ddbd8-93ab-4d45-bdee-8dd49e88ad5a None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:21.341536 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:21.344268 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 373/761] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 10:35:21 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 27 10:35:21.357224 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcb81a0-2519-438f-81a3-5258266db633 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:21.358126 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcb81a0-2519-438f-81a3-5258266db633 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:21.358582 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcb81a0-2519-438f-81a3-5258266db633 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:21.359049 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcb81a0-2519-438f-81a3-5258266db633 None admin] PATH_INFO: `/v3/users/nova` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:21.367441 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdcb81a0-2519-438f-81a3-5258266db633 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:21.369089 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdcb81a0-2519-438f-81a3-5258266db633 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:21.380751 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application [None req-fdcb81a0-2519-438f-81a3-5258266db633 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 27 10:35:21.380751 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:35:21.380751 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:35:21.380751 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:35:21.380751 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:35:21.380751 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:35:21.380751 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:35:21.380751 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:35:21.380751 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:35:21.380751 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:35:21.380751 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:35:21.380751 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:35:21.380751 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 10:35:21.380751 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 10:35:21.380751 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 10:35:21.380751 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 10:35:21.382168 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:35:21.382168 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:35:21.382168 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 10:35:21.382168 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 10:35:21.382168 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 10:35:21.382168 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 10:35:21.382168 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:35:21.382168 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:35:21.382168 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:35:21.382168 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:35:21.382168 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:35:21.382168 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application with Lock( Jul 27 10:35:21.382168 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:35:21.382168 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:35:21.382168 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:35:21.382168 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:35:21.382168 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:35:21.382168 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:35:21.383119 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:35:21.383119 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:35:21.383119 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 10:35:21.383119 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 10:35:21.383119 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 10:35:21.383119 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 10:35:21.383119 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 10:35:21.383119 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 10:35:21.383119 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 27 10:35:21.383119 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Jul 27 10:35:21.385557 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 389/762] 10.222.0.36 () {56 vars in 1093 bytes} [Wed Jul 27 10:35:21 2022] GET /identity/v3/users/nova => generated 82 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 10:35:21.390735 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6143c097-8354-43cf-ba9b-3df34d19273f None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:21.390885 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcb81a0-2519-438f-81a3-5258266db633 None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:21.493616 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6143c097-8354-43cf-ba9b-3df34d19273f None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:21.521552 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:21.538666 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee917843-6e0d-4843-8d9e-b9101ec71b01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:21.539681 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee917843-6e0d-4843-8d9e-b9101ec71b01 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:21.540197 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee917843-6e0d-4843-8d9e-b9101ec71b01 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:21.540676 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee917843-6e0d-4843-8d9e-b9101ec71b01 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:21.541513 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee917843-6e0d-4843-8d9e-b9101ec71b01 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:21.543186 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee917843-6e0d-4843-8d9e-b9101ec71b01 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:21.560760 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application [None req-ee917843-6e0d-4843-8d9e-b9101ec71b01 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 27 10:35:21.560760 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:35:21.560760 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 27 10:35:21.560760 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 27 10:35:21.560760 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 27 10:35:21.560760 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 27 10:35:21.560760 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 27 10:35:21.560760 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 10:35:21.560760 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 27 10:35:21.560760 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Jul 27 10:35:21.560760 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 10:35:21.560760 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Jul 27 10:35:21.560760 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:35:21.560760 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:35:21.560760 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:35:21.560760 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:35:21.561697 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:35:21.561697 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:35:21.561697 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:35:21.561697 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:35:21.561697 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:35:21.561697 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:35:21.561697 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:35:21.561697 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 27 10:35:21.561697 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 27 10:35:21.561697 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 27 10:35:21.561697 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 27 10:35:21.561697 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:35:21.561697 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:35:21.561697 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:35:21.561697 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:35:21.561697 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:35:21.562536 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:35:21.562536 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:35:21.562536 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application with Lock( Jul 27 10:35:21.562536 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:35:21.562536 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:35:21.562536 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:35:21.562536 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:35:21.562536 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:35:21.562536 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:35:21.562536 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:35:21.562536 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:35:21.562536 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 27 10:35:21.562536 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 10:35:21.562536 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 27 10:35:21.562536 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Jul 27 10:35:21.571115 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcb81a0-2519-438f-81a3-5258266db633 None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:21.572886 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 374/763] 10.222.0.36 () {56 vars in 1129 bytes} [Wed Jul 27 10:35:21 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 10:35:21.578868 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee917843-6e0d-4843-8d9e-b9101ec71b01 None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:21.612899 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f26ea0d-783c-40ab-bdc6-24e7a15c5c2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:21.613445 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f26ea0d-783c-40ab-bdc6-24e7a15c5c2b None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:21.614654 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f26ea0d-783c-40ab-bdc6-24e7a15c5c2b None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:21.615668 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f26ea0d-783c-40ab-bdc6-24e7a15c5c2b None admin] PATH_INFO: `/v3/users` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:21.616478 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f26ea0d-783c-40ab-bdc6-24e7a15c5c2b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86827) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:35:21.616783 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f26ea0d-783c-40ab-bdc6-24e7a15c5c2b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:21.618442 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f26ea0d-783c-40ab-bdc6-24e7a15c5c2b None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:21.654068 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 390/764] 10.222.0.36 () {56 vars in 1133 bytes} [Wed Jul 27 10:35:21 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 10:35:21.658286 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f26ea0d-783c-40ab-bdc6-24e7a15c5c2b None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:21.784133 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:21.792724 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee917843-6e0d-4843-8d9e-b9101ec71b01 None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:21.828909 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0c0dbf-d72e-4d5a-a3a4-7a4656fe4e05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:21.829912 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0c0dbf-d72e-4d5a-a3a4-7a4656fe4e05 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:21.830411 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0c0dbf-d72e-4d5a-a3a4-7a4656fe4e05 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:21.830946 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0c0dbf-d72e-4d5a-a3a4-7a4656fe4e05 None admin] PATH_INFO: `/v3/services` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:21.831870 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0c0dbf-d72e-4d5a-a3a4-7a4656fe4e05 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=86828) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:35:21.832387 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0c0dbf-d72e-4d5a-a3a4-7a4656fe4e05 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:21.834068 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0c0dbf-d72e-4d5a-a3a4-7a4656fe4e05 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:21.844176 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 375/765] 10.222.0.36 () {56 vars in 1124 bytes} [Wed Jul 27 10:35:21 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 10:35:21.845667 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0c0dbf-d72e-4d5a-a3a4-7a4656fe4e05 None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:21.901556 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f26ea0d-783c-40ab-bdc6-24e7a15c5c2b None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:21.907305 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:21.967913 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0c0dbf-d72e-4d5a-a3a4-7a4656fe4e05 None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:21.974736 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4f10d1-0d37-4b17-b760-601bc6417cc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:21.975477 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4f10d1-0d37-4b17-b760-601bc6417cc1 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:21.975761 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4f10d1-0d37-4b17-b760-601bc6417cc1 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:21.976040 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4f10d1-0d37-4b17-b760-601bc6417cc1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:22.000807 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4f10d1-0d37-4b17-b760-601bc6417cc1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:22.002332 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4f10d1-0d37-4b17-b760-601bc6417cc1 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:22.006059 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c42d33-bce2-4b21-a25d-de921aa6becf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:22.007608 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c42d33-bce2-4b21-a25d-de921aa6becf None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:22.008147 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c42d33-bce2-4b21-a25d-de921aa6becf None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:22.008691 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c42d33-bce2-4b21-a25d-de921aa6becf None admin] PATH_INFO: `/v3/services` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:22.009861 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c42d33-bce2-4b21-a25d-de921aa6becf None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=86828) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:35:22.010392 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c42d33-bce2-4b21-a25d-de921aa6becf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:22.012091 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c42d33-bce2-4b21-a25d-de921aa6becf None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:22.022737 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 376/766] 10.222.0.36 () {56 vars in 1124 bytes} [Wed Jul 27 10:35:21 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:35:22.027560 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c42d33-bce2-4b21-a25d-de921aa6becf None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:22.035605 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application [None req-2f4f10d1-0d37-4b17-b760-601bc6417cc1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 10:35:22.035605 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:35:22.035605 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:35:22.035605 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:35:22.035605 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:35:22.035605 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:35:22.035605 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:35:22.035605 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:35:22.035605 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:35:22.035605 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:35:22.035605 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:35:22.035605 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:35:22.035605 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 10:35:22.035605 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 10:35:22.035605 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 10:35:22.036545 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 10:35:22.036545 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:35:22.036545 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:35:22.036545 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:35:22.036545 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:35:22.036545 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:35:22.036545 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:35:22.036545 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:35:22.036545 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application with Lock( Jul 27 10:35:22.036545 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:35:22.036545 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:35:22.036545 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:35:22.036545 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:35:22.036545 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:35:22.036545 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:35:22.036545 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:35:22.036545 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:35:22.037544 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 10:35:22.037544 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 10:35:22.037544 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 10:35:22.037544 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Jul 27 10:35:22.040803 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 391/767] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 10:35:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 383 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 10:35:22.048475 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4f10d1-0d37-4b17-b760-601bc6417cc1 None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:22.218176 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c42d33-bce2-4b21-a25d-de921aa6becf None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:22.259377 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:22.260894 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:22.264630 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4f10d1-0d37-4b17-b760-601bc6417cc1 None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:22.292419 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c0f1a0-7d03-4b4f-acce-f97d1f6c474d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:22.293061 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c0f1a0-7d03-4b4f-acce-f97d1f6c474d None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:22.293344 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c0f1a0-7d03-4b4f-acce-f97d1f6c474d None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:22.293617 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c0f1a0-7d03-4b4f-acce-f97d1f6c474d None admin] PATH_INFO: `/v3/endpoints` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:22.294381 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c0f1a0-7d03-4b4f-acce-f97d1f6c474d None admin] RBAC: Adding query filter params (interface=public, service_id=306ac01ebd1d41469ad8ca36368270d8, region_id=RegionOne) {{(pid=86828) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:35:22.294739 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c0f1a0-7d03-4b4f-acce-f97d1f6c474d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:22.296123 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c0f1a0-7d03-4b4f-acce-f97d1f6c474d None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:22.304155 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 377/768] 10.222.0.36 () {56 vars in 1251 bytes} [Wed Jul 27 10:35:22 2022] GET /identity/v3/endpoints?service_id=306ac01ebd1d41469ad8ca36368270d8&interface=public®ion_id=RegionOne => generated 194 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:35:22.305441 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d13b40-bd55-42db-9d01-bdfde4a80fbd None None] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:22.305665 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d13b40-bd55-42db-9d01-bdfde4a80fbd None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:22.305947 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d13b40-bd55-42db-9d01-bdfde4a80fbd None None] PATH_INFO: `/` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:22.306838 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed489f70-0f24-4ce7-8a6d-7845d9a49898 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:22.307182 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 378/769] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 10:35:22 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 10:35:22.307453 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed489f70-0f24-4ce7-8a6d-7845d9a49898 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:22.307734 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed489f70-0f24-4ce7-8a6d-7845d9a49898 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:22.308003 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed489f70-0f24-4ce7-8a6d-7845d9a49898 None admin] PATH_INFO: `/v3/domains` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:22.308700 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed489f70-0f24-4ce7-8a6d-7845d9a49898 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86827) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:35:22.309003 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed489f70-0f24-4ce7-8a6d-7845d9a49898 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:22.310422 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed489f70-0f24-4ce7-8a6d-7845d9a49898 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:22.312423 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51ee856-d2f7-46bc-81ec-e0b62ad76157 None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:22.312716 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51ee856-d2f7-46bc-81ec-e0b62ad76157 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:22.312996 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51ee856-d2f7-46bc-81ec-e0b62ad76157 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:22.325569 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 392/770] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 10:35:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:35:22.326987 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed489f70-0f24-4ce7-8a6d-7845d9a49898 None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:22.336218 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-a51ee856-d2f7-46bc-81ec-e0b62ad76157 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:35:22.337417 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-a51ee856-d2f7-46bc-81ec-e0b62ad76157 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:35:22.411034 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 379/771] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 10:35:22 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 27 10:35:22.411947 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51ee856-d2f7-46bc-81ec-e0b62ad76157 None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:22.455293 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed489f70-0f24-4ce7-8a6d-7845d9a49898 None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:22.507303 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b16b3b-48c9-4e60-88e8-bda9310269c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:22.508132 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b16b3b-48c9-4e60-88e8-bda9310269c2 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:22.508409 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b16b3b-48c9-4e60-88e8-bda9310269c2 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:22.508679 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b16b3b-48c9-4e60-88e8-bda9310269c2 None admin] PATH_INFO: `/v3/services` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:22.509892 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b16b3b-48c9-4e60-88e8-bda9310269c2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:22.511999 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b16b3b-48c9-4e60-88e8-bda9310269c2 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:22.519687 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 393/772] 10.222.0.36 () {56 vars in 1087 bytes} [Wed Jul 27 10:35:22 2022] GET /identity/v3/services => generated 772 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:35:22.521106 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b16b3b-48c9-4e60-88e8-bda9310269c2 None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:22.558003 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51ee856-d2f7-46bc-81ec-e0b62ad76157 None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:22.558515 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:22.575247 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:22.599851 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a135ad-85b8-48a7-9d22-d59bff3e6cb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:22.600597 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a135ad-85b8-48a7-9d22-d59bff3e6cb0 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:22.600878 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a135ad-85b8-48a7-9d22-d59bff3e6cb0 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:22.601150 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a135ad-85b8-48a7-9d22-d59bff3e6cb0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:22.609576 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a135ad-85b8-48a7-9d22-d59bff3e6cb0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:22.611213 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a135ad-85b8-48a7-9d22-d59bff3e6cb0 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:22.625243 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application [None req-a2a135ad-85b8-48a7-9d22-d59bff3e6cb0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 10:35:22.625243 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:35:22.625243 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:35:22.625243 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:35:22.625243 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:35:22.625243 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:35:22.625243 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:35:22.625243 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:35:22.625243 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:35:22.625243 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:35:22.625243 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:35:22.625243 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:35:22.625243 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 10:35:22.625243 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 10:35:22.625243 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 10:35:22.626341 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 10:35:22.626341 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:35:22.626341 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:35:22.626341 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:35:22.626341 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:35:22.626341 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:35:22.626341 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:35:22.626341 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:35:22.626341 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application with Lock( Jul 27 10:35:22.626341 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:35:22.626341 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:35:22.626341 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:35:22.626341 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:35:22.626341 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:35:22.626341 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:35:22.626341 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:35:22.626341 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:35:22.627297 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 10:35:22.627297 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 10:35:22.627297 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 10:35:22.627297 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 10:35:22.627297 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 10:35:22.627297 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 10:35:22.627297 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 10:35:22.627297 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Jul 27 10:35:22.627297 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 380/773] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Jul 27 10:35:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 10:35:22.632644 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a135ad-85b8-48a7-9d22-d59bff3e6cb0 None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:22.664022 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b16b3b-48c9-4e60-88e8-bda9310269c2 None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:22.704334 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b401abe-bef6-45b5-9b84-80d0cbc760ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:22.705401 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b401abe-bef6-45b5-9b84-80d0cbc760ea None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:22.705891 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b401abe-bef6-45b5-9b84-80d0cbc760ea None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:22.706370 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b401abe-bef6-45b5-9b84-80d0cbc760ea None admin] PATH_INFO: `/v3/services/network` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:22.707279 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b401abe-bef6-45b5-9b84-80d0cbc760ea None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:22.709162 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b401abe-bef6-45b5-9b84-80d0cbc760ea None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:22.719826 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application [None req-3b401abe-bef6-45b5-9b84-80d0cbc760ea None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 27 10:35:22.719826 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:35:22.719826 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 27 10:35:22.719826 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 27 10:35:22.719826 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 27 10:35:22.719826 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 27 10:35:22.719826 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 27 10:35:22.719826 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 10:35:22.719826 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 27 10:35:22.719826 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Jul 27 10:35:22.719826 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 10:35:22.719826 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Jul 27 10:35:22.719826 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:35:22.719826 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:35:22.719826 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:35:22.719826 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:35:22.719826 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:35:22.720843 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:35:22.720843 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:35:22.720843 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:35:22.720843 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:35:22.720843 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:35:22.720843 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:35:22.720843 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 27 10:35:22.720843 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 27 10:35:22.720843 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 27 10:35:22.720843 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 27 10:35:22.720843 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:35:22.720843 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:35:22.720843 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:35:22.720843 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:35:22.720843 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:35:22.720843 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:35:22.721666 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:35:22.721666 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application with Lock( Jul 27 10:35:22.721666 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:35:22.721666 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:35:22.721666 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:35:22.721666 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:35:22.721666 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:35:22.721666 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:35:22.721666 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:35:22.721666 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:35:22.721666 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 27 10:35:22.721666 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 10:35:22.721666 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 27 10:35:22.721666 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Jul 27 10:35:22.726702 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 394/774] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Jul 27 10:35:22 2022] GET /identity/v3/services/network => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 10:35:22.728364 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7434e5e-cdae-4c83-bbe3-95d357340e16 None None] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:22.728882 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7434e5e-cdae-4c83-bbe3-95d357340e16 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:22.729379 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7434e5e-cdae-4c83-bbe3-95d357340e16 None None] PATH_INFO: `/` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:22.731516 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 395/775] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 10:35:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 10:35:22.732450 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7434e5e-cdae-4c83-bbe3-95d357340e16 None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:22.780286 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a135ad-85b8-48a7-9d22-d59bff3e6cb0 None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:22.819702 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa0c7ae-6b02-468c-ace6-af9c176bdd42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:22.821171 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa0c7ae-6b02-468c-ace6-af9c176bdd42 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:22.821697 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa0c7ae-6b02-468c-ace6-af9c176bdd42 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:22.822175 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa0c7ae-6b02-468c-ace6-af9c176bdd42 None admin] PATH_INFO: `/v3/projects` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:22.823299 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa0c7ae-6b02-468c-ace6-af9c176bdd42 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86828) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:35:22.823677 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa0c7ae-6b02-468c-ace6-af9c176bdd42 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:22.825242 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa0c7ae-6b02-468c-ace6-af9c176bdd42 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:22.840751 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:22.842347 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 381/776] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Jul 27 10:35:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:35:22.843770 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963cda2b-4b8d-4e4e-958a-2fc315355140 None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:22.844161 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963cda2b-4b8d-4e4e-958a-2fc315355140 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:22.844478 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963cda2b-4b8d-4e4e-958a-2fc315355140 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:22.856732 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7434e5e-cdae-4c83-bbe3-95d357340e16 None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:22.866485 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-963cda2b-4b8d-4e4e-958a-2fc315355140 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:35:22.867814 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-963cda2b-4b8d-4e4e-958a-2fc315355140 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:35:22.898375 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:22.903589 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced031c2-9017-40c6-bd28-6921e4139079 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:22.905013 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced031c2-9017-40c6-bd28-6921e4139079 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:22.905605 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced031c2-9017-40c6-bd28-6921e4139079 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:22.906079 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced031c2-9017-40c6-bd28-6921e4139079 None admin] PATH_INFO: `/v3/services` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:22.906998 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced031c2-9017-40c6-bd28-6921e4139079 None admin] RBAC: Adding query filter params (name=network) {{(pid=86827) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:35:22.907790 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced031c2-9017-40c6-bd28-6921e4139079 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:22.910342 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced031c2-9017-40c6-bd28-6921e4139079 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:22.917185 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 396/777] 10.222.0.36 () {56 vars in 1112 bytes} [Wed Jul 27 10:35:22 2022] GET /identity/v3/services?name=network => generated 124 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:35:22.918673 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced031c2-9017-40c6-bd28-6921e4139079 None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:22.948669 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 382/778] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 10:35:22 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 10:35:22.949534 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963cda2b-4b8d-4e4e-958a-2fc315355140 None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:23.059464 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963cda2b-4b8d-4e4e-958a-2fc315355140 None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:23.065444 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced031c2-9017-40c6-bd28-6921e4139079 None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:23.095975 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d25621-22e1-45e9-9eb1-f8e55085f8bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:23.096603 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d25621-22e1-45e9-9eb1-f8e55085f8bd None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:23.096873 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d25621-22e1-45e9-9eb1-f8e55085f8bd None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:23.097145 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d25621-22e1-45e9-9eb1-f8e55085f8bd None admin] PATH_INFO: `/v3/services` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:23.097821 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d25621-22e1-45e9-9eb1-f8e55085f8bd None admin] RBAC: Adding query filter params (type=network) {{(pid=86828) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:35:23.098123 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d25621-22e1-45e9-9eb1-f8e55085f8bd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:23.099515 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d25621-22e1-45e9-9eb1-f8e55085f8bd None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:23.105587 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 383/779] 10.222.0.36 () {56 vars in 1112 bytes} [Wed Jul 27 10:35:22 2022] GET /identity/v3/services?type=network => generated 350 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:35:23.106480 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d25621-22e1-45e9-9eb1-f8e55085f8bd None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:23.111696 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f70de7-c3db-4835-ac31-b06c3f0e6fef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:23.112716 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f70de7-c3db-4835-ac31-b06c3f0e6fef None admin] REQUEST_METHOD: `PUT` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:23.113197 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f70de7-c3db-4835-ac31-b06c3f0e6fef None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:23.113672 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f70de7-c3db-4835-ac31-b06c3f0e6fef None admin] PATH_INFO: `/v3/projects/5f296d8e3b8f4001909addc49d8dac01/users/e059d8b6c28c4fcba925d05d7c98e4ac/roles/21156560eb6b436bb567a40b4976f3f4` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:23.116422 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f70de7-c3db-4835-ac31-b06c3f0e6fef None admin] RBAC: Authorizing `identity:create_grant(project_id=5f296d8e3b8f4001909addc49d8dac01, user_id=e059d8b6c28c4fcba925d05d7c98e4ac, role_id=21156560eb6b436bb567a40b4976f3f4)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:23.119048 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f70de7-c3db-4835-ac31-b06c3f0e6fef None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:23.136810 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:23.142392 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 397/780] 10.222.0.36 () {58 vars in 1439 bytes} [Wed Jul 27 10:35:22 2022] PUT /identity/v3/projects/5f296d8e3b8f4001909addc49d8dac01/users/e059d8b6c28c4fcba925d05d7c98e4ac/roles/21156560eb6b436bb567a40b4976f3f4 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:35:23.143631 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f70de7-c3db-4835-ac31-b06c3f0e6fef None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:23.166861 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:23.222015 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d25621-22e1-45e9-9eb1-f8e55085f8bd None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:23.263361 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d803c2-6a4f-4276-93bc-a3a3de47fdd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:23.264705 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d803c2-6a4f-4276-93bc-a3a3de47fdd6 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:23.265264 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d803c2-6a4f-4276-93bc-a3a3de47fdd6 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:23.266199 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d803c2-6a4f-4276-93bc-a3a3de47fdd6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:23.281822 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d803c2-6a4f-4276-93bc-a3a3de47fdd6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 10:35:23.282013 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f70de7-c3db-4835-ac31-b06c3f0e6fef None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:23.289430 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application [None req-e0d803c2-6a4f-4276-93bc-a3a3de47fdd6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 27 10:35:23.289430 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:35:23.289430 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:35:23.289430 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:35:23.289430 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:35:23.289430 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:35:23.289430 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:35:23.289430 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:35:23.289430 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:35:23.289430 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:35:23.289430 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:35:23.289430 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:35:23.289430 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 10:35:23.289430 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 10:35:23.289430 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 10:35:23.289430 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 10:35:23.290432 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 10:35:23.290432 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 10:35:23.290432 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 10:35:23.290432 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 10:35:23.290432 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:35:23.290432 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:35:23.290432 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:35:23.290432 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:35:23.290432 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:35:23.290432 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:35:23.290432 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:35:23.290432 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application with Lock( Jul 27 10:35:23.290432 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:35:23.290432 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:35:23.290432 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:35:23.290432 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:35:23.291320 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:35:23.291320 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:35:23.291320 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:35:23.291320 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:35:23.291320 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 10:35:23.291320 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 10:35:23.291320 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 10:35:23.291320 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 10:35:23.291320 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 10:35:23.291320 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 10:35:23.291320 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 27 10:35:23.291320 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Jul 27 10:35:23.293198 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 384/781] 10.222.0.36 () {56 vars in 1102 bytes} [Wed Jul 27 10:35:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 10:35:23.298206 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d803c2-6a4f-4276-93bc-a3a3de47fdd6 None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:23.342141 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40cbbd2a-25cb-489e-b779-401b83d02e0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:23.342798 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cbbd2a-25cb-489e-b779-401b83d02e0d None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:23.343135 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cbbd2a-25cb-489e-b779-401b83d02e0d None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:23.343408 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cbbd2a-25cb-489e-b779-401b83d02e0d None admin] PATH_INFO: `/v3/endpoints` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:23.344647 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40cbbd2a-25cb-489e-b779-401b83d02e0d None admin] RBAC: Adding query filter params (interface=public, service_id=c9b80393ff6643b1bd5b441357089368, region_id=RegionOne) {{(pid=86827) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:35:23.344647 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40cbbd2a-25cb-489e-b779-401b83d02e0d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:23.345924 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40cbbd2a-25cb-489e-b779-401b83d02e0d None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:23.369887 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 398/782] 10.222.0.36 () {56 vars in 1251 bytes} [Wed Jul 27 10:35:23 2022] GET /identity/v3/endpoints?service_id=c9b80393ff6643b1bd5b441357089368&interface=public®ion_id=RegionOne => generated 194 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:35:23.376305 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40cbbd2a-25cb-489e-b779-401b83d02e0d None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:23.414050 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:23.427702 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d803c2-6a4f-4276-93bc-a3a3de47fdd6 None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:23.464629 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fb464a-2822-4af8-8f3d-745305c585cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:23.465581 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fb464a-2822-4af8-8f3d-745305c585cc None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:23.466006 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fb464a-2822-4af8-8f3d-745305c585cc None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:23.466429 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fb464a-2822-4af8-8f3d-745305c585cc None admin] PATH_INFO: `/v3/roles` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:23.467317 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fb464a-2822-4af8-8f3d-745305c585cc None admin] RBAC: Adding query filter params (name=service) {{(pid=86828) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:35:23.467789 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fb464a-2822-4af8-8f3d-745305c585cc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:23.469304 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fb464a-2822-4af8-8f3d-745305c585cc None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:23.477260 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:23.478905 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 385/783] 10.222.0.36 () {56 vars in 1103 bytes} [Wed Jul 27 10:35:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 10:35:23.481073 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937b691c-92c7-4ac2-85d9-7b7cdfa289a6 None None] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:23.481576 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937b691c-92c7-4ac2-85d9-7b7cdfa289a6 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:23.482070 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937b691c-92c7-4ac2-85d9-7b7cdfa289a6 None None] PATH_INFO: `/` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:23.483667 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 386/784] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 10:35:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 10:35:23.484605 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937b691c-92c7-4ac2-85d9-7b7cdfa289a6 None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:23.518163 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40cbbd2a-25cb-489e-b779-401b83d02e0d None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:23.569324 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a150ea98-c2d9-4d01-80ad-23fcaa6feec5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:23.570047 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a150ea98-c2d9-4d01-80ad-23fcaa6feec5 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:23.570334 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a150ea98-c2d9-4d01-80ad-23fcaa6feec5 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:23.570706 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a150ea98-c2d9-4d01-80ad-23fcaa6feec5 None admin] PATH_INFO: `/v3/services` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:23.571842 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a150ea98-c2d9-4d01-80ad-23fcaa6feec5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:23.573719 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a150ea98-c2d9-4d01-80ad-23fcaa6feec5 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:23.581553 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 399/785] 10.222.0.36 () {56 vars in 1087 bytes} [Wed Jul 27 10:35:23 2022] GET /identity/v3/services => generated 772 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:35:23.582881 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320b6b67-541f-43bc-83cc-c0adf2515326 None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:23.583221 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320b6b67-541f-43bc-83cc-c0adf2515326 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:23.583507 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320b6b67-541f-43bc-83cc-c0adf2515326 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:23.601037 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937b691c-92c7-4ac2-85d9-7b7cdfa289a6 None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:23.608662 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-320b6b67-541f-43bc-83cc-c0adf2515326 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:35:23.610639 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-320b6b67-541f-43bc-83cc-c0adf2515326 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:35:23.656695 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bbe802-3ec0-41f8-b42a-9ef43d1c5384 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:23.657750 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bbe802-3ec0-41f8-b42a-9ef43d1c5384 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:23.658264 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bbe802-3ec0-41f8-b42a-9ef43d1c5384 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:23.658786 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bbe802-3ec0-41f8-b42a-9ef43d1c5384 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:23.677201 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18bbe802-3ec0-41f8-b42a-9ef43d1c5384 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:23.679874 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18bbe802-3ec0-41f8-b42a-9ef43d1c5384 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:23.712544 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application [None req-18bbe802-3ec0-41f8-b42a-9ef43d1c5384 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 10:35:23.712544 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:35:23.712544 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:35:23.712544 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:35:23.712544 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:35:23.712544 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:35:23.712544 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:35:23.712544 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:35:23.712544 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:35:23.712544 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:35:23.712544 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:35:23.712544 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:35:23.712544 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 10:35:23.712544 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 10:35:23.712544 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 10:35:23.713507 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 10:35:23.713507 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:35:23.713507 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:35:23.713507 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:35:23.713507 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:35:23.713507 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:35:23.713507 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:35:23.713507 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:35:23.713507 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application with Lock( Jul 27 10:35:23.713507 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:35:23.713507 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:35:23.713507 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:35:23.713507 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:35:23.713507 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:35:23.713507 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:35:23.713507 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:35:23.713507 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:35:23.714418 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 10:35:23.714418 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 10:35:23.714418 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 10:35:23.714418 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Jul 27 10:35:23.717644 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 387/786] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 10:35:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 10:35:23.736459 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bbe802-3ec0-41f8-b42a-9ef43d1c5384 None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:23.740019 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:23.752861 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 400/787] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 10:35:23 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 10:35:23.781515 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320b6b67-541f-43bc-83cc-c0adf2515326 None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:23.873405 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:23.877895 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bbe802-3ec0-41f8-b42a-9ef43d1c5384 None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:23.910722 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320b6b67-541f-43bc-83cc-c0adf2515326 None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:23.915636 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06d7d9f-a2b0-4948-9c00-7a001172a374 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:23.920903 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06d7d9f-a2b0-4948-9c00-7a001172a374 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:23.920903 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06d7d9f-a2b0-4948-9c00-7a001172a374 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:23.921235 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06d7d9f-a2b0-4948-9c00-7a001172a374 None admin] PATH_INFO: `/v3/domains` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:23.924124 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06d7d9f-a2b0-4948-9c00-7a001172a374 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86828) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:35:23.924124 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06d7d9f-a2b0-4948-9c00-7a001172a374 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:23.924124 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06d7d9f-a2b0-4948-9c00-7a001172a374 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:23.937798 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 388/788] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 10:35:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:35:23.942920 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06d7d9f-a2b0-4948-9c00-7a001172a374 None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:23.972238 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd5fdf8-4a62-4868-86f6-abb4776dd374 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:23.972922 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd5fdf8-4a62-4868-86f6-abb4776dd374 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:23.973203 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd5fdf8-4a62-4868-86f6-abb4776dd374 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:23.973480 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd5fdf8-4a62-4868-86f6-abb4776dd374 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:24.000426 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd5fdf8-4a62-4868-86f6-abb4776dd374 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 10:35:24.010237 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application [None req-9dd5fdf8-4a62-4868-86f6-abb4776dd374 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 10:35:24.010237 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:35:24.010237 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:35:24.010237 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:35:24.010237 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:35:24.010237 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:35:24.010237 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:35:24.010237 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:35:24.010237 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:35:24.010237 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:35:24.010237 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:35:24.010237 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:35:24.010237 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 10:35:24.010237 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 10:35:24.010237 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 10:35:24.010237 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 10:35:24.011219 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 10:35:24.011219 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 10:35:24.011219 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 10:35:24.011219 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 10:35:24.011219 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:35:24.011219 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:35:24.011219 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:35:24.011219 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:35:24.011219 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:35:24.011219 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:35:24.011219 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:35:24.011219 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application with Lock( Jul 27 10:35:24.011219 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:35:24.011219 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:35:24.011219 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:35:24.011219 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:35:24.012031 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:35:24.012031 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:35:24.012031 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:35:24.012031 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:35:24.012031 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 10:35:24.012031 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 10:35:24.012031 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 10:35:24.012031 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 10:35:24.012031 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 10:35:24.012031 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 10:35:24.012031 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 10:35:24.012031 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Jul 27 10:35:24.015191 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 401/789] 10.222.0.36 () {56 vars in 1096 bytes} [Wed Jul 27 10:35:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 10:35:24.020632 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd5fdf8-4a62-4868-86f6-abb4776dd374 None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:24.079687 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:24.158931 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06d7d9f-a2b0-4948-9c00-7a001172a374 None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:24.209908 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd5fdf8-4a62-4868-86f6-abb4776dd374 None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:24.245379 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf39db03-6d14-4285-8e57-4d320b58a1c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:24.246037 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf39db03-6d14-4285-8e57-4d320b58a1c6 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:24.246322 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf39db03-6d14-4285-8e57-4d320b58a1c6 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:24.246616 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf39db03-6d14-4285-8e57-4d320b58a1c6 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:24.259189 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf39db03-6d14-4285-8e57-4d320b58a1c6 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:24.260681 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:24.261761 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf39db03-6d14-4285-8e57-4d320b58a1c6 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:24.276128 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application [None req-bf39db03-6d14-4285-8e57-4d320b58a1c6 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 27 10:35:24.276128 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:35:24.276128 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:35:24.276128 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:35:24.276128 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:35:24.276128 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:35:24.276128 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:35:24.276128 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:35:24.276128 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:35:24.276128 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:35:24.276128 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:35:24.276128 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:35:24.276128 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 10:35:24.276128 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 10:35:24.276128 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 10:35:24.276128 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 10:35:24.277108 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:35:24.277108 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:35:24.277108 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 10:35:24.277108 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 10:35:24.277108 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 10:35:24.277108 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 10:35:24.277108 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:35:24.277108 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:35:24.277108 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:35:24.277108 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:35:24.277108 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:35:24.277108 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application with Lock( Jul 27 10:35:24.277108 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:35:24.277108 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:35:24.277108 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:35:24.277108 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:35:24.277108 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:35:24.277108 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:35:24.278133 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:35:24.278133 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:35:24.278133 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 10:35:24.278133 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 10:35:24.278133 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 10:35:24.278133 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 10:35:24.278133 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 10:35:24.278133 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 10:35:24.278133 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 27 10:35:24.278133 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Jul 27 10:35:24.280845 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9d2858-1400-4a28-87cb-455d77d74d28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:24.281316 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 389/790] 10.222.0.36 () {56 vars in 1117 bytes} [Wed Jul 27 10:35:23 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 339 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 10:35:24.281419 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9d2858-1400-4a28-87cb-455d77d74d28 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:24.281613 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9d2858-1400-4a28-87cb-455d77d74d28 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:24.281887 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9d2858-1400-4a28-87cb-455d77d74d28 None admin] PATH_INFO: `/v3/roles` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:24.282633 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9d2858-1400-4a28-87cb-455d77d74d28 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86827) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:35:24.282940 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9d2858-1400-4a28-87cb-455d77d74d28 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:24.284341 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9d2858-1400-4a28-87cb-455d77d74d28 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:24.286446 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf39db03-6d14-4285-8e57-4d320b58a1c6 None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:24.304708 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 402/791] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jul 27 10:35:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 10:35:24.323680 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9d2858-1400-4a28-87cb-455d77d74d28 None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:24.408632 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf39db03-6d14-4285-8e57-4d320b58a1c6 None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:24.434920 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:24.450372 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866c8306-4b4d-4571-85a9-954eb4fbb50b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:24.452007 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866c8306-4b4d-4571-85a9-954eb4fbb50b None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:24.452699 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866c8306-4b4d-4571-85a9-954eb4fbb50b None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:24.453438 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866c8306-4b4d-4571-85a9-954eb4fbb50b None admin] PATH_INFO: `/v3/users` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:24.455021 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866c8306-4b4d-4571-85a9-954eb4fbb50b None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=86828) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:35:24.455825 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866c8306-4b4d-4571-85a9-954eb4fbb50b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:24.458573 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866c8306-4b4d-4571-85a9-954eb4fbb50b None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:24.478954 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 390/792] 10.222.0.36 () {56 vars in 1149 bytes} [Wed Jul 27 10:35:24 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 10:35:24.486465 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866c8306-4b4d-4571-85a9-954eb4fbb50b None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:24.564863 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9d2858-1400-4a28-87cb-455d77d74d28 None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:24.636353 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866c8306-4b4d-4571-85a9-954eb4fbb50b None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:24.673721 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6d29e8-732d-4b89-b338-6c68380d2cc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:24.674743 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6d29e8-732d-4b89-b338-6c68380d2cc8 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:24.675259 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6d29e8-732d-4b89-b338-6c68380d2cc8 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:24.676051 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6d29e8-732d-4b89-b338-6c68380d2cc8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:24.688221 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6d29e8-732d-4b89-b338-6c68380d2cc8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:24.689923 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6d29e8-732d-4b89-b338-6c68380d2cc8 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:24.706124 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application [None req-bf6d29e8-732d-4b89-b338-6c68380d2cc8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 10:35:24.706124 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:35:24.706124 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:35:24.706124 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:35:24.706124 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:35:24.706124 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:35:24.706124 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:35:24.706124 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:35:24.706124 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:35:24.706124 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:35:24.706124 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:35:24.706124 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:35:24.706124 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 10:35:24.706124 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 10:35:24.706124 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 10:35:24.707082 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 10:35:24.707082 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:35:24.707082 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:35:24.707082 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:35:24.707082 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:35:24.707082 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:35:24.707082 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:35:24.707082 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:35:24.707082 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application with Lock( Jul 27 10:35:24.707082 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:35:24.707082 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:35:24.707082 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:35:24.707082 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:35:24.707082 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:35:24.707082 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:35:24.707082 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:35:24.707082 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:35:24.707961 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 10:35:24.707961 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 10:35:24.707961 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 10:35:24.707961 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Jul 27 10:35:24.710229 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 391/793] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 10:35:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 10:35:24.716395 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6d29e8-732d-4b89-b338-6c68380d2cc8 None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:24.721475 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb925789-b51b-4737-bad8-eea5c69c7bd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:24.721968 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb925789-b51b-4737-bad8-eea5c69c7bd9 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:24.722246 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb925789-b51b-4737-bad8-eea5c69c7bd9 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:24.722521 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb925789-b51b-4737-bad8-eea5c69c7bd9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:24.741922 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb925789-b51b-4737-bad8-eea5c69c7bd9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:24.743385 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb925789-b51b-4737-bad8-eea5c69c7bd9 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:24.767667 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application [None req-eb925789-b51b-4737-bad8-eea5c69c7bd9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 10:35:24.767667 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:35:24.767667 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:35:24.767667 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:35:24.767667 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:35:24.767667 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:35:24.767667 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:35:24.767667 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:35:24.767667 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:35:24.767667 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:35:24.767667 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:35:24.767667 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:35:24.767667 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 10:35:24.767667 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 10:35:24.767667 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 10:35:24.768575 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 10:35:24.768575 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:35:24.768575 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:35:24.768575 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:35:24.768575 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:35:24.768575 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:35:24.768575 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:35:24.768575 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:35:24.768575 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application with Lock( Jul 27 10:35:24.768575 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:35:24.768575 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:35:24.768575 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:35:24.768575 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:35:24.768575 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:35:24.768575 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:35:24.768575 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:35:24.768575 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:35:24.769473 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 10:35:24.769473 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 10:35:24.769473 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 10:35:24.769473 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Jul 27 10:35:24.770105 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 403/794] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 10:35:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 450 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 10:35:24.777755 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb925789-b51b-4737-bad8-eea5c69c7bd9 None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:24.782935 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:24.863465 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:24.908275 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6d29e8-732d-4b89-b338-6c68380d2cc8 None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:24.938469 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb925789-b51b-4737-bad8-eea5c69c7bd9 None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:24.971573 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416f21ca-d323-46d5-bd2e-801c5bb9128f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:24.972212 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416f21ca-d323-46d5-bd2e-801c5bb9128f None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:24.972492 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416f21ca-d323-46d5-bd2e-801c5bb9128f None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:24.972767 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416f21ca-d323-46d5-bd2e-801c5bb9128f None admin] PATH_INFO: `/v3/domains` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:24.973469 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416f21ca-d323-46d5-bd2e-801c5bb9128f None admin] RBAC: Adding query filter params (name=Default) {{(pid=86828) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:35:24.973778 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416f21ca-d323-46d5-bd2e-801c5bb9128f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:24.975236 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416f21ca-d323-46d5-bd2e-801c5bb9128f None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:24.988700 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 392/795] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 10:35:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 10:35:24.993559 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71af6e6-19ec-4918-b829-e1f3647e11e9 None None] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:24.993559 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71af6e6-19ec-4918-b829-e1f3647e11e9 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:24.993559 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71af6e6-19ec-4918-b829-e1f3647e11e9 None None] PATH_INFO: `/` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:24.995759 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 393/796] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 10:35:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 10:35:24.996510 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71af6e6-19ec-4918-b829-e1f3647e11e9 None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:24.997158 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2da53e5-35e5-430d-88f1-b34d38ae0979 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:24.998006 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2da53e5-35e5-430d-88f1-b34d38ae0979 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:24.998482 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2da53e5-35e5-430d-88f1-b34d38ae0979 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:24.998980 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2da53e5-35e5-430d-88f1-b34d38ae0979 None admin] PATH_INFO: `/v3/domains` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:25.000243 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2da53e5-35e5-430d-88f1-b34d38ae0979 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86827) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:35:25.001733 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2da53e5-35e5-430d-88f1-b34d38ae0979 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:25.003319 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2da53e5-35e5-430d-88f1-b34d38ae0979 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:25.023488 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 404/797] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 10:35:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:35:25.024953 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708d8e3f-ca23-43ef-b055-ff886972caf0 None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:25.025314 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708d8e3f-ca23-43ef-b055-ff886972caf0 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:25.025662 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708d8e3f-ca23-43ef-b055-ff886972caf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:25.046788 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-708d8e3f-ca23-43ef-b055-ff886972caf0 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:35:25.048101 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-708d8e3f-ca23-43ef-b055-ff886972caf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:35:25.117267 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:25.133221 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71af6e6-19ec-4918-b829-e1f3647e11e9 None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:25.137634 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 405/798] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 10:35:25 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 10:35:25.137634 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708d8e3f-ca23-43ef-b055-ff886972caf0 None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:25.186005 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b92e0e7-4f14-410e-89f8-360f5cfc05f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:25.186680 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b92e0e7-4f14-410e-89f8-360f5cfc05f6 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:25.186967 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b92e0e7-4f14-410e-89f8-360f5cfc05f6 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:25.187245 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b92e0e7-4f14-410e-89f8-360f5cfc05f6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:25.188654 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:25.200149 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b92e0e7-4f14-410e-89f8-360f5cfc05f6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:25.202481 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b92e0e7-4f14-410e-89f8-360f5cfc05f6 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:25.218763 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application [None req-1b92e0e7-4f14-410e-89f8-360f5cfc05f6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 10:35:25.218763 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:35:25.218763 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:35:25.218763 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:35:25.218763 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:35:25.218763 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:35:25.218763 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:35:25.218763 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:35:25.218763 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:35:25.218763 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:35:25.218763 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:35:25.218763 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:35:25.218763 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 10:35:25.218763 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 10:35:25.218763 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 10:35:25.219825 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 10:35:25.219825 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:35:25.219825 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:35:25.219825 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:35:25.219825 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:35:25.219825 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:35:25.219825 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:35:25.219825 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:35:25.219825 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application with Lock( Jul 27 10:35:25.219825 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:35:25.219825 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:35:25.219825 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:35:25.219825 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:35:25.219825 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:35:25.219825 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:35:25.219825 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:35:25.219825 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:35:25.220898 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 10:35:25.220898 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 10:35:25.220898 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 10:35:25.220898 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 10:35:25.220898 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 10:35:25.220898 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 10:35:25.220898 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 10:35:25.220898 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Jul 27 10:35:25.223805 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 394/799] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Jul 27 10:35:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 10:35:25.224827 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b92e0e7-4f14-410e-89f8-360f5cfc05f6 None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:25.269576 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708d8e3f-ca23-43ef-b055-ff886972caf0 None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:25.307750 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65454447-72a4-4179-94c6-119810e1efeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:25.308622 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65454447-72a4-4179-94c6-119810e1efeb None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:25.308953 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65454447-72a4-4179-94c6-119810e1efeb None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:25.309305 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65454447-72a4-4179-94c6-119810e1efeb None admin] PATH_INFO: `/v3/users/swift` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:25.317361 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65454447-72a4-4179-94c6-119810e1efeb None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:25.319083 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65454447-72a4-4179-94c6-119810e1efeb None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:25.331930 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application [None req-65454447-72a4-4179-94c6-119810e1efeb None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 27 10:35:25.331930 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:35:25.331930 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:35:25.331930 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:35:25.331930 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:35:25.331930 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:35:25.331930 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:35:25.331930 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:35:25.331930 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:35:25.331930 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:35:25.331930 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:35:25.331930 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:35:25.331930 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 10:35:25.331930 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 10:35:25.331930 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 10:35:25.331930 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 10:35:25.333153 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:35:25.333153 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:35:25.333153 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 10:35:25.333153 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 10:35:25.333153 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 10:35:25.333153 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 10:35:25.333153 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:35:25.333153 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:35:25.333153 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:35:25.333153 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:35:25.333153 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:35:25.333153 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application with Lock( Jul 27 10:35:25.333153 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:35:25.333153 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:35:25.333153 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:35:25.333153 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:35:25.333153 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:35:25.333153 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:35:25.334840 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:35:25.334840 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:35:25.334840 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 10:35:25.334840 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 10:35:25.334840 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 10:35:25.334840 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 10:35:25.334840 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 10:35:25.334840 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 10:35:25.334840 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 27 10:35:25.334840 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Jul 27 10:35:25.338991 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b92e0e7-4f14-410e-89f8-360f5cfc05f6 None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:25.340639 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 406/800] 10.222.0.36 () {56 vars in 1096 bytes} [Wed Jul 27 10:35:25 2022] GET /identity/v3/users/swift => generated 83 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 10:35:25.341684 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65454447-72a4-4179-94c6-119810e1efeb None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:25.379794 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61287666-ec71-49a4-90c8-eb32b2570545 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:25.380680 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61287666-ec71-49a4-90c8-eb32b2570545 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:25.381156 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61287666-ec71-49a4-90c8-eb32b2570545 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:25.381570 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61287666-ec71-49a4-90c8-eb32b2570545 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:25.382341 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61287666-ec71-49a4-90c8-eb32b2570545 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:25.384136 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61287666-ec71-49a4-90c8-eb32b2570545 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:25.393888 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application [None req-61287666-ec71-49a4-90c8-eb32b2570545 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 27 10:35:25.393888 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:35:25.393888 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 27 10:35:25.393888 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 27 10:35:25.393888 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 27 10:35:25.393888 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 27 10:35:25.393888 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 27 10:35:25.393888 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 10:35:25.393888 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 27 10:35:25.393888 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Jul 27 10:35:25.393888 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 10:35:25.393888 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Jul 27 10:35:25.393888 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:35:25.393888 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:35:25.393888 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:35:25.393888 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:35:25.394854 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:35:25.394854 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:35:25.394854 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:35:25.394854 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:35:25.394854 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:35:25.394854 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:35:25.394854 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:35:25.394854 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 27 10:35:25.394854 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 27 10:35:25.394854 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 27 10:35:25.394854 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 27 10:35:25.394854 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:35:25.394854 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:35:25.394854 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:35:25.394854 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:35:25.394854 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:35:25.395702 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:35:25.395702 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:35:25.395702 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application with Lock( Jul 27 10:35:25.395702 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:35:25.395702 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:35:25.395702 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:35:25.395702 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:35:25.395702 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:35:25.395702 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:35:25.395702 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:35:25.395702 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:35:25.395702 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 27 10:35:25.395702 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 10:35:25.395702 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 27 10:35:25.395702 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Jul 27 10:35:25.403525 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 395/801] 10.222.0.36 () {56 vars in 1129 bytes} [Wed Jul 27 10:35:25 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 10:35:25.404666 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61287666-ec71-49a4-90c8-eb32b2570545 None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:25.425228 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:25.446491 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:25.452597 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65454447-72a4-4179-94c6-119810e1efeb None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:25.486592 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240dadf1-6476-4ead-ac5f-f2bcc68cfc90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:25.487238 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240dadf1-6476-4ead-ac5f-f2bcc68cfc90 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:25.487526 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240dadf1-6476-4ead-ac5f-f2bcc68cfc90 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:25.487807 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240dadf1-6476-4ead-ac5f-f2bcc68cfc90 None admin] PATH_INFO: `/v3/projects` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:25.488568 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240dadf1-6476-4ead-ac5f-f2bcc68cfc90 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86827) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:35:25.488880 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240dadf1-6476-4ead-ac5f-f2bcc68cfc90 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:25.490370 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240dadf1-6476-4ead-ac5f-f2bcc68cfc90 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:25.502461 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 407/802] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Jul 27 10:35:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:35:25.503303 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240dadf1-6476-4ead-ac5f-f2bcc68cfc90 None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:25.518520 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61287666-ec71-49a4-90c8-eb32b2570545 None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:25.555533 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7ce8b3-c29c-4fe9-be2c-9f1b7049f3e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:25.556146 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7ce8b3-c29c-4fe9-be2c-9f1b7049f3e7 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:25.556423 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7ce8b3-c29c-4fe9-be2c-9f1b7049f3e7 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:25.556697 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7ce8b3-c29c-4fe9-be2c-9f1b7049f3e7 None admin] PATH_INFO: `/v3/users` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:25.557672 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7ce8b3-c29c-4fe9-be2c-9f1b7049f3e7 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=86828) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:35:25.557985 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7ce8b3-c29c-4fe9-be2c-9f1b7049f3e7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:25.559614 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7ce8b3-c29c-4fe9-be2c-9f1b7049f3e7 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:25.578402 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 396/803] 10.222.0.36 () {56 vars in 1135 bytes} [Wed Jul 27 10:35:25 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:35:25.579896 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b574db7-a82a-432d-a59d-0e13a849be93 None None] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:25.580192 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b574db7-a82a-432d-a59d-0e13a849be93 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:25.580462 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b574db7-a82a-432d-a59d-0e13a849be93 None None] PATH_INFO: `/` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:25.581795 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 397/804] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 10:35:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 10:35:25.582381 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b574db7-a82a-432d-a59d-0e13a849be93 None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:25.620085 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240dadf1-6476-4ead-ac5f-f2bcc68cfc90 None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:25.655383 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e74221-30de-47b2-b07b-24faa68dcd4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:25.656010 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e74221-30de-47b2-b07b-24faa68dcd4d None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:25.656297 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e74221-30de-47b2-b07b-24faa68dcd4d None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:25.656579 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e74221-30de-47b2-b07b-24faa68dcd4d None admin] PATH_INFO: `/v3/services` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:25.657263 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e74221-30de-47b2-b07b-24faa68dcd4d None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=86827) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:35:25.657574 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e74221-30de-47b2-b07b-24faa68dcd4d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:25.658980 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e74221-30de-47b2-b07b-24faa68dcd4d None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:25.664033 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 408/805] 10.222.0.36 () {56 vars in 1124 bytes} [Wed Jul 27 10:35:25 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:35:25.664807 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e74221-30de-47b2-b07b-24faa68dcd4d None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:25.666683 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:25.689637 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:25.697375 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b574db7-a82a-432d-a59d-0e13a849be93 None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:25.731680 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab21a2f3-1a30-4684-a77e-f98fed0ce9f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:25.732359 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab21a2f3-1a30-4684-a77e-f98fed0ce9f5 None admin] REQUEST_METHOD: `PUT` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:25.732643 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab21a2f3-1a30-4684-a77e-f98fed0ce9f5 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:25.732934 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab21a2f3-1a30-4684-a77e-f98fed0ce9f5 None admin] PATH_INFO: `/v3/projects/5f296d8e3b8f4001909addc49d8dac01/users/3d9fed24b4ba44b0b5961a603c4362c4/roles/9e0038c6bbd34556af3f84f9e2a7c2c5` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:25.748411 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab21a2f3-1a30-4684-a77e-f98fed0ce9f5 None admin] RBAC: Authorizing `identity:create_grant(project_id=5f296d8e3b8f4001909addc49d8dac01, user_id=3d9fed24b4ba44b0b5961a603c4362c4, role_id=9e0038c6bbd34556af3f84f9e2a7c2c5)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:25.749913 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab21a2f3-1a30-4684-a77e-f98fed0ce9f5 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:25.764406 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 398/806] 10.222.0.36 () {58 vars in 1439 bytes} [Wed Jul 27 10:35:25 2022] PUT /identity/v3/projects/5f296d8e3b8f4001909addc49d8dac01/users/3d9fed24b4ba44b0b5961a603c4362c4/roles/9e0038c6bbd34556af3f84f9e2a7c2c5 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:35:25.765632 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e865357-241e-4fa5-93a4-18bd7d167d68 None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:25.765923 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e865357-241e-4fa5-93a4-18bd7d167d68 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:25.766202 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e865357-241e-4fa5-93a4-18bd7d167d68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:25.785924 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-5e865357-241e-4fa5-93a4-18bd7d167d68 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:35:25.787418 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-5e865357-241e-4fa5-93a4-18bd7d167d68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:35:25.793142 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e74221-30de-47b2-b07b-24faa68dcd4d None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:25.836469 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1838aa1-8597-4ed2-aa1e-a10ade4d0022 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:25.837390 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1838aa1-8597-4ed2-aa1e-a10ade4d0022 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:25.837918 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1838aa1-8597-4ed2-aa1e-a10ade4d0022 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:25.838335 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1838aa1-8597-4ed2-aa1e-a10ade4d0022 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:25.851452 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1838aa1-8597-4ed2-aa1e-a10ade4d0022 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:25.853156 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1838aa1-8597-4ed2-aa1e-a10ade4d0022 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:25.872076 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application [None req-a1838aa1-8597-4ed2-aa1e-a10ade4d0022 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 10:35:25.872076 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:35:25.872076 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:35:25.872076 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:35:25.872076 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:35:25.872076 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:35:25.872076 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:35:25.872076 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:35:25.872076 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:35:25.872076 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:35:25.872076 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:35:25.872076 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:35:25.872076 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 10:35:25.872076 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 10:35:25.872076 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 10:35:25.873037 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 10:35:25.873037 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:35:25.873037 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:35:25.873037 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:35:25.873037 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:35:25.873037 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:35:25.873037 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:35:25.873037 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:35:25.873037 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application with Lock( Jul 27 10:35:25.873037 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:35:25.873037 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:35:25.873037 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:35:25.873037 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:35:25.873037 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:35:25.873037 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:35:25.873037 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:35:25.873037 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:35:25.873927 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 10:35:25.873927 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 10:35:25.873927 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 10:35:25.873927 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Jul 27 10:35:25.878127 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 409/807] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 10:35:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 10:35:25.879180 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1838aa1-8597-4ed2-aa1e-a10ade4d0022 None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:25.879957 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 399/808] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 10:35:25 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 10:35:25.881895 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e865357-241e-4fa5-93a4-18bd7d167d68 None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:25.963413 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:25.974500 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:26.008718 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1838aa1-8597-4ed2-aa1e-a10ade4d0022 None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:26.022875 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e865357-241e-4fa5-93a4-18bd7d167d68 None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:26.042453 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a79322-4ce1-46d3-b8c4-95af7dadf74a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:26.043098 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a79322-4ce1-46d3-b8c4-95af7dadf74a None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:26.043377 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a79322-4ce1-46d3-b8c4-95af7dadf74a None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:26.043650 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a79322-4ce1-46d3-b8c4-95af7dadf74a None admin] PATH_INFO: `/v3/services` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:26.044330 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a79322-4ce1-46d3-b8c4-95af7dadf74a None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=86827) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:35:26.044633 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a79322-4ce1-46d3-b8c4-95af7dadf74a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:26.045986 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a79322-4ce1-46d3-b8c4-95af7dadf74a None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:26.051745 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 410/809] 10.222.0.36 () {56 vars in 1124 bytes} [Wed Jul 27 10:35:25 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:35:26.052515 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a79322-4ce1-46d3-b8c4-95af7dadf74a None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:26.065273 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ba1a3e-9370-420b-bd4c-49c91e56f4fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:26.066229 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ba1a3e-9370-420b-bd4c-49c91e56f4fe None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:26.066695 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ba1a3e-9370-420b-bd4c-49c91e56f4fe None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:26.067165 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ba1a3e-9370-420b-bd4c-49c91e56f4fe None admin] PATH_INFO: `/v3/domains` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:26.068021 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ba1a3e-9370-420b-bd4c-49c91e56f4fe None admin] RBAC: Adding query filter params (name=Default) {{(pid=86828) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:35:26.069176 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ba1a3e-9370-420b-bd4c-49c91e56f4fe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:26.070918 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ba1a3e-9370-420b-bd4c-49c91e56f4fe None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:26.083664 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 400/810] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 10:35:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:35:26.085692 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842cad8a-b49c-4990-90f8-3d628164bda9 None None] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:26.085982 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842cad8a-b49c-4990-90f8-3d628164bda9 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:26.086264 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842cad8a-b49c-4990-90f8-3d628164bda9 None None] PATH_INFO: `/` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:26.087655 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 401/811] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 10:35:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 10:35:26.088193 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842cad8a-b49c-4990-90f8-3d628164bda9 None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:26.163302 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a79322-4ce1-46d3-b8c4-95af7dadf74a None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:26.196518 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842cad8a-b49c-4990-90f8-3d628164bda9 None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:26.200943 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c69e23-c82e-443a-b292-91bddea41a3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:26.201594 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c69e23-c82e-443a-b292-91bddea41a3b None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:26.201878 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c69e23-c82e-443a-b292-91bddea41a3b None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:26.202156 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c69e23-c82e-443a-b292-91bddea41a3b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:26.204311 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:26.211951 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:26.214224 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c69e23-c82e-443a-b292-91bddea41a3b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 10:35:26.219606 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application [None req-75c69e23-c82e-443a-b292-91bddea41a3b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 10:35:26.219606 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:35:26.219606 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:35:26.219606 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:35:26.219606 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:35:26.219606 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:35:26.219606 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:35:26.219606 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:35:26.219606 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:35:26.219606 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:35:26.219606 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:35:26.219606 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:35:26.219606 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 10:35:26.219606 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 10:35:26.219606 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 10:35:26.219606 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 10:35:26.220555 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 10:35:26.220555 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 10:35:26.220555 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 10:35:26.220555 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 10:35:26.220555 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:35:26.220555 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:35:26.220555 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:35:26.220555 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:35:26.220555 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:35:26.220555 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:35:26.220555 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:35:26.220555 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application with Lock( Jul 27 10:35:26.220555 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:35:26.220555 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:35:26.220555 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:35:26.220555 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:35:26.221388 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:35:26.221388 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:35:26.221388 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:35:26.221388 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:35:26.221388 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 10:35:26.221388 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 10:35:26.221388 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 10:35:26.221388 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 10:35:26.221388 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 10:35:26.221388 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 10:35:26.221388 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 10:35:26.221388 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Jul 27 10:35:26.221388 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 411/812] 10.222.0.36 () {56 vars in 1096 bytes} [Wed Jul 27 10:35:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 10:35:26.222085 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c69e23-c82e-443a-b292-91bddea41a3b None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:26.241291 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07071a2b-7bd3-4ce8-8dcc-46dc13a017af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:26.241921 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07071a2b-7bd3-4ce8-8dcc-46dc13a017af None admin] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:26.242197 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07071a2b-7bd3-4ce8-8dcc-46dc13a017af None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:26.242466 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07071a2b-7bd3-4ce8-8dcc-46dc13a017af None admin] PATH_INFO: `/v3/endpoints` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:26.243205 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07071a2b-7bd3-4ce8-8dcc-46dc13a017af None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:26.244575 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07071a2b-7bd3-4ce8-8dcc-46dc13a017af None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:26.270693 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 402/813] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Jul 27 10:35:26 2022] POST /identity/v3/endpoints => generated 346 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 10:35:26.272834 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f5bb8b-ec15-46b1-91ae-160d438512ae None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:26.273134 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f5bb8b-ec15-46b1-91ae-160d438512ae None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:26.273421 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f5bb8b-ec15-46b1-91ae-160d438512ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:26.299292 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-22f5bb8b-ec15-46b1-91ae-160d438512ae None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:35:26.301354 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-22f5bb8b-ec15-46b1-91ae-160d438512ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:35:26.340397 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c69e23-c82e-443a-b292-91bddea41a3b None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:26.385956 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116f66fe-3c60-412b-9918-117a3afd9ce1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:26.387105 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116f66fe-3c60-412b-9918-117a3afd9ce1 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:26.387608 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116f66fe-3c60-412b-9918-117a3afd9ce1 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:26.388115 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116f66fe-3c60-412b-9918-117a3afd9ce1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:26.396132 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116f66fe-3c60-412b-9918-117a3afd9ce1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:26.406638 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 403/814] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 10:35:26 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 10:35:26.406638 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f5bb8b-ec15-46b1-91ae-160d438512ae None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:26.411051 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116f66fe-3c60-412b-9918-117a3afd9ce1 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:26.430703 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application [None req-116f66fe-3c60-412b-9918-117a3afd9ce1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 10:35:26.430703 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:35:26.430703 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:35:26.430703 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:35:26.430703 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:35:26.430703 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:35:26.430703 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:35:26.430703 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:35:26.430703 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:35:26.430703 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:35:26.430703 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:35:26.430703 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:35:26.430703 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 10:35:26.430703 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 10:35:26.430703 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 10:35:26.433056 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 10:35:26.433056 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:35:26.433056 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:35:26.433056 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:35:26.433056 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:35:26.433056 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:35:26.433056 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:35:26.433056 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:35:26.433056 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application with Lock( Jul 27 10:35:26.433056 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:35:26.433056 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:35:26.433056 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:35:26.433056 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:35:26.433056 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:35:26.433056 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:35:26.433056 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:35:26.433056 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:35:26.433953 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 10:35:26.433953 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 10:35:26.433953 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 10:35:26.433953 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 10:35:26.433953 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 10:35:26.433953 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 10:35:26.433953 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 10:35:26.433953 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Jul 27 10:35:26.433953 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 412/815] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Jul 27 10:35:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 10:35:26.433953 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116f66fe-3c60-412b-9918-117a3afd9ce1 None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:26.496023 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:26.521232 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:26.524843 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f5bb8b-ec15-46b1-91ae-160d438512ae None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:26.563003 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cb37fd-2742-4cd0-bb64-ab050af60f02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:26.563790 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cb37fd-2742-4cd0-bb64-ab050af60f02 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:26.564285 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cb37fd-2742-4cd0-bb64-ab050af60f02 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:26.564630 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cb37fd-2742-4cd0-bb64-ab050af60f02 None admin] PATH_INFO: `/v3/roles` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:26.565411 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1cb37fd-2742-4cd0-bb64-ab050af60f02 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86828) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:35:26.565780 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1cb37fd-2742-4cd0-bb64-ab050af60f02 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:26.567082 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116f66fe-3c60-412b-9918-117a3afd9ce1 None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:26.567330 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1cb37fd-2742-4cd0-bb64-ab050af60f02 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:26.576995 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 404/816] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jul 27 10:35:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:35:26.577937 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cb37fd-2742-4cd0-bb64-ab050af60f02 None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:26.608651 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876befa4-830b-471d-a07d-8a3be1cee683 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:26.609293 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876befa4-830b-471d-a07d-8a3be1cee683 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:26.609569 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876befa4-830b-471d-a07d-8a3be1cee683 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:26.609846 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876befa4-830b-471d-a07d-8a3be1cee683 None admin] PATH_INFO: `/v3/services/network` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:26.610473 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876befa4-830b-471d-a07d-8a3be1cee683 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:26.612042 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876befa4-830b-471d-a07d-8a3be1cee683 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:26.621455 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application [None req-876befa4-830b-471d-a07d-8a3be1cee683 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 27 10:35:26.621455 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:35:26.621455 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 27 10:35:26.621455 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 27 10:35:26.621455 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 27 10:35:26.621455 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 27 10:35:26.621455 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 27 10:35:26.621455 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 10:35:26.621455 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 27 10:35:26.621455 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Jul 27 10:35:26.621455 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 10:35:26.621455 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Jul 27 10:35:26.621455 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:35:26.621455 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:35:26.621455 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:35:26.621455 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:35:26.621455 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:35:26.622428 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:35:26.622428 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:35:26.622428 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:35:26.622428 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:35:26.622428 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:35:26.622428 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:35:26.622428 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 27 10:35:26.622428 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 27 10:35:26.622428 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 27 10:35:26.622428 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 27 10:35:26.622428 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:35:26.622428 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:35:26.622428 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:35:26.622428 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:35:26.622428 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:35:26.622428 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:35:26.623292 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:35:26.623292 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application with Lock( Jul 27 10:35:26.623292 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:35:26.623292 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:35:26.623292 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:35:26.623292 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:35:26.623292 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:35:26.623292 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:35:26.623292 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:35:26.623292 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:35:26.623292 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 27 10:35:26.623292 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 10:35:26.623292 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 27 10:35:26.623292 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Jul 27 10:35:26.623292 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 413/817] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Jul 27 10:35:26 2022] GET /identity/v3/services/network => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 10:35:26.624083 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876befa4-830b-471d-a07d-8a3be1cee683 None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:26.691519 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cb37fd-2742-4cd0-bb64-ab050af60f02 None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:26.727218 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876befa4-830b-471d-a07d-8a3be1cee683 None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:26.729609 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a9e15d-1620-4d3b-a1ec-df5a6e55b0e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:26.730244 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a9e15d-1620-4d3b-a1ec-df5a6e55b0e8 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:26.730530 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a9e15d-1620-4d3b-a1ec-df5a6e55b0e8 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:26.730838 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a9e15d-1620-4d3b-a1ec-df5a6e55b0e8 None admin] PATH_INFO: `/v3/projects` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:26.731597 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a9e15d-1620-4d3b-a1ec-df5a6e55b0e8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86828) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:35:26.731913 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a9e15d-1620-4d3b-a1ec-df5a6e55b0e8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:26.733346 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a9e15d-1620-4d3b-a1ec-df5a6e55b0e8 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:26.741188 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:26.747781 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 405/818] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Jul 27 10:35:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:35:26.748818 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a9e15d-1620-4d3b-a1ec-df5a6e55b0e8 None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:26.769484 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53b00ff-f207-4825-9c82-e5bb8869081e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:26.770132 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53b00ff-f207-4825-9c82-e5bb8869081e None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:26.770410 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53b00ff-f207-4825-9c82-e5bb8869081e None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:26.770714 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53b00ff-f207-4825-9c82-e5bb8869081e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:26.779177 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:26.783818 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53b00ff-f207-4825-9c82-e5bb8869081e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:26.785360 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53b00ff-f207-4825-9c82-e5bb8869081e None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:26.801277 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application [None req-e53b00ff-f207-4825-9c82-e5bb8869081e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 10:35:26.801277 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:35:26.801277 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:35:26.801277 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:35:26.801277 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:35:26.801277 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:35:26.801277 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:35:26.801277 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:35:26.801277 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:35:26.801277 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:35:26.801277 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:35:26.801277 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:35:26.801277 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 10:35:26.801277 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 10:35:26.801277 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 10:35:26.802392 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 10:35:26.802392 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:35:26.802392 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:35:26.802392 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:35:26.802392 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:35:26.802392 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:35:26.802392 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:35:26.802392 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:35:26.802392 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application with Lock( Jul 27 10:35:26.802392 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:35:26.802392 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:35:26.802392 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:35:26.802392 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:35:26.802392 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:35:26.802392 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:35:26.802392 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:35:26.802392 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:35:26.803877 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 10:35:26.803877 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 10:35:26.803877 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 10:35:26.803877 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Jul 27 10:35:26.803877 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 414/819] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 10:35:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 10:35:26.803877 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53b00ff-f207-4825-9c82-e5bb8869081e None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:26.868773 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a9e15d-1620-4d3b-a1ec-df5a6e55b0e8 None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:26.906273 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90bddc8-e227-44d2-bca4-936a63ab760d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:26.907185 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90bddc8-e227-44d2-bca4-936a63ab760d None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:26.907721 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90bddc8-e227-44d2-bca4-936a63ab760d None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:26.908148 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90bddc8-e227-44d2-bca4-936a63ab760d None admin] PATH_INFO: `/v3/services` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:26.909079 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90bddc8-e227-44d2-bca4-936a63ab760d None admin] RBAC: Adding query filter params (name=network) {{(pid=86828) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:35:26.909672 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90bddc8-e227-44d2-bca4-936a63ab760d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:26.911246 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90bddc8-e227-44d2-bca4-936a63ab760d None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:26.913046 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53b00ff-f207-4825-9c82-e5bb8869081e None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:26.917382 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 406/820] 10.222.0.36 () {56 vars in 1112 bytes} [Wed Jul 27 10:35:26 2022] GET /identity/v3/services?name=network => generated 124 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:35:26.918386 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90bddc8-e227-44d2-bca4-936a63ab760d None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:26.961603 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6315053d-33d3-4476-83af-f55bfd17a139 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:26.962369 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6315053d-33d3-4476-83af-f55bfd17a139 None admin] REQUEST_METHOD: `PUT` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:26.962773 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6315053d-33d3-4476-83af-f55bfd17a139 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:26.963116 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6315053d-33d3-4476-83af-f55bfd17a139 None admin] PATH_INFO: `/v3/projects/5f296d8e3b8f4001909addc49d8dac01/users/89ac506eefcd4e42a84b44fa15fb3975/roles/21156560eb6b436bb567a40b4976f3f4` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:26.965409 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6315053d-33d3-4476-83af-f55bfd17a139 None admin] RBAC: Authorizing `identity:create_grant(project_id=5f296d8e3b8f4001909addc49d8dac01, user_id=89ac506eefcd4e42a84b44fa15fb3975, role_id=21156560eb6b436bb567a40b4976f3f4)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:26.966744 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6315053d-33d3-4476-83af-f55bfd17a139 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:26.986792 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 415/821] 10.222.0.36 () {58 vars in 1439 bytes} [Wed Jul 27 10:35:26 2022] PUT /identity/v3/projects/5f296d8e3b8f4001909addc49d8dac01/users/89ac506eefcd4e42a84b44fa15fb3975/roles/21156560eb6b436bb567a40b4976f3f4 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:35:26.987818 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6315053d-33d3-4476-83af-f55bfd17a139 None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:27.027192 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:27.038256 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90bddc8-e227-44d2-bca4-936a63ab760d None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:27.042667 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:27.108124 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f708e7c3-e8ef-4cd5-ab95-76385564b921 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:27.108775 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f708e7c3-e8ef-4cd5-ab95-76385564b921 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:27.109054 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f708e7c3-e8ef-4cd5-ab95-76385564b921 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:27.109329 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f708e7c3-e8ef-4cd5-ab95-76385564b921 None admin] PATH_INFO: `/v3/domains` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:27.110103 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f708e7c3-e8ef-4cd5-ab95-76385564b921 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86828) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:35:27.110413 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f708e7c3-e8ef-4cd5-ab95-76385564b921 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:27.112074 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f708e7c3-e8ef-4cd5-ab95-76385564b921 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:27.140542 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 407/822] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 10:35:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:35:27.145665 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f708e7c3-e8ef-4cd5-ab95-76385564b921 None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:27.196824 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6315053d-33d3-4476-83af-f55bfd17a139 None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:27.255498 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820ac921-fbf8-4fdc-abee-e2ec5d57e7d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:27.256522 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820ac921-fbf8-4fdc-abee-e2ec5d57e7d5 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:27.257091 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820ac921-fbf8-4fdc-abee-e2ec5d57e7d5 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:27.257551 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820ac921-fbf8-4fdc-abee-e2ec5d57e7d5 None admin] PATH_INFO: `/v3/services` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:27.258528 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820ac921-fbf8-4fdc-abee-e2ec5d57e7d5 None admin] RBAC: Adding query filter params (type=network) {{(pid=86827) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:35:27.259345 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820ac921-fbf8-4fdc-abee-e2ec5d57e7d5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:27.261167 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820ac921-fbf8-4fdc-abee-e2ec5d57e7d5 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:27.267266 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 416/823] 10.222.0.36 () {56 vars in 1112 bytes} [Wed Jul 27 10:35:26 2022] GET /identity/v3/services?type=network => generated 350 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:35:27.274757 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f708e7c3-e8ef-4cd5-ab95-76385564b921 None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:27.276628 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820ac921-fbf8-4fdc-abee-e2ec5d57e7d5 None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:27.314001 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7da8dee-bb9f-4b41-b595-03bd03aa67e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:27.315254 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7da8dee-bb9f-4b41-b595-03bd03aa67e2 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:27.315988 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7da8dee-bb9f-4b41-b595-03bd03aa67e2 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:27.316672 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7da8dee-bb9f-4b41-b595-03bd03aa67e2 None admin] PATH_INFO: `/v3/users/nova` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:27.328733 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7da8dee-bb9f-4b41-b595-03bd03aa67e2 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:27.331635 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7da8dee-bb9f-4b41-b595-03bd03aa67e2 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:27.348328 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application [None req-b7da8dee-bb9f-4b41-b595-03bd03aa67e2 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 27 10:35:27.348328 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:35:27.348328 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:35:27.348328 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:35:27.348328 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:35:27.348328 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:35:27.348328 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:35:27.348328 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:35:27.348328 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:35:27.348328 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:35:27.348328 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:35:27.348328 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:35:27.348328 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 10:35:27.348328 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 10:35:27.348328 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 10:35:27.348328 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 10:35:27.349465 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:35:27.349465 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:35:27.349465 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 10:35:27.349465 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 10:35:27.349465 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 10:35:27.349465 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 10:35:27.349465 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:35:27.349465 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:35:27.349465 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:35:27.349465 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:35:27.349465 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:35:27.349465 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application with Lock( Jul 27 10:35:27.349465 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:35:27.349465 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:35:27.349465 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:35:27.349465 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:35:27.349465 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:35:27.349465 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:35:27.351215 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:35:27.351215 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:35:27.351215 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 10:35:27.351215 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 10:35:27.351215 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 10:35:27.351215 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 10:35:27.351215 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 10:35:27.351215 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 10:35:27.351215 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 27 10:35:27.351215 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Jul 27 10:35:27.354402 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 408/824] 10.222.0.36 () {56 vars in 1093 bytes} [Wed Jul 27 10:35:27 2022] GET /identity/v3/users/nova => generated 82 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 10:35:27.367546 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7da8dee-bb9f-4b41-b595-03bd03aa67e2 None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:27.377204 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:27.396491 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:27.407853 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820ac921-fbf8-4fdc-abee-e2ec5d57e7d5 None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:27.449831 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f82717-8d4f-4210-818e-1ca955ff80df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:27.450518 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f82717-8d4f-4210-818e-1ca955ff80df None admin] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:27.450835 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f82717-8d4f-4210-818e-1ca955ff80df None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:27.451109 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f82717-8d4f-4210-818e-1ca955ff80df None admin] PATH_INFO: `/v3/endpoints` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:27.452441 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f82717-8d4f-4210-818e-1ca955ff80df None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:27.453816 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f82717-8d4f-4210-818e-1ca955ff80df None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:27.494803 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 417/825] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Jul 27 10:35:27 2022] POST /identity/v3/endpoints => generated 327 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 10:35:27.522699 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7da8dee-bb9f-4b41-b595-03bd03aa67e2 None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:27.569144 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305dc096-9fcb-494a-a066-f6a9b22742ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:27.569719 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305dc096-9fcb-494a-a066-f6a9b22742ee None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:27.570057 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305dc096-9fcb-494a-a066-f6a9b22742ee None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:27.570338 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305dc096-9fcb-494a-a066-f6a9b22742ee None admin] PATH_INFO: `/v3/users` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:27.571236 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305dc096-9fcb-494a-a066-f6a9b22742ee None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86828) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:35:27.571548 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305dc096-9fcb-494a-a066-f6a9b22742ee None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:27.573228 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305dc096-9fcb-494a-a066-f6a9b22742ee None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:27.595354 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 409/826] 10.222.0.36 () {56 vars in 1133 bytes} [Wed Jul 27 10:35:27 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:35:27.601835 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f82717-8d4f-4210-818e-1ca955ff80df None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:27.726698 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f82717-8d4f-4210-818e-1ca955ff80df None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:27.762790 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6341f67-c922-47bb-87ae-e4e955da5bee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:27.763426 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6341f67-c922-47bb-87ae-e4e955da5bee None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:27.763714 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6341f67-c922-47bb-87ae-e4e955da5bee None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:27.763994 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6341f67-c922-47bb-87ae-e4e955da5bee None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:27.781752 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:27.841190 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6341f67-c922-47bb-87ae-e4e955da5bee None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:27.842622 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6341f67-c922-47bb-87ae-e4e955da5bee None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:27.972789 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 418/827] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jul 27 10:35:27 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 27 10:35:27.978891 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305dc096-9fcb-494a-a066-f6a9b22742ee None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:28.037357 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f1ce9a-a914-4d34-90f7-49c6ccc94eeb None None] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:28.038375 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f1ce9a-a914-4d34-90f7-49c6ccc94eeb None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:28.038593 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f1ce9a-a914-4d34-90f7-49c6ccc94eeb None None] PATH_INFO: `/` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:28.040081 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 419/828] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 10:35:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 10:35:28.045259 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a278fd-ccdf-4f0a-a6c9-bf618906020d None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:28.045802 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a278fd-ccdf-4f0a-a6c9-bf618906020d None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:28.046299 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a278fd-ccdf-4f0a-a6c9-bf618906020d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:28.059373 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:28.066247 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-d0a278fd-ccdf-4f0a-a6c9-bf618906020d None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:35:28.068178 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-d0a278fd-ccdf-4f0a-a6c9-bf618906020d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:35:28.103484 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305dc096-9fcb-494a-a066-f6a9b22742ee None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:28.117652 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:28.137779 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef59f05b-d223-45bb-b8b2-6cf9f95dd7a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:28.138686 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef59f05b-d223-45bb-b8b2-6cf9f95dd7a6 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:28.139130 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef59f05b-d223-45bb-b8b2-6cf9f95dd7a6 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:28.139405 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef59f05b-d223-45bb-b8b2-6cf9f95dd7a6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:28.144553 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 420/829] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 10:35:28 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 27 10:35:28.150264 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef59f05b-d223-45bb-b8b2-6cf9f95dd7a6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:28.151742 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef59f05b-d223-45bb-b8b2-6cf9f95dd7a6 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:28.167533 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application [None req-ef59f05b-d223-45bb-b8b2-6cf9f95dd7a6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 10:35:28.167533 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:35:28.167533 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:35:28.167533 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:35:28.167533 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:35:28.167533 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:35:28.167533 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:35:28.167533 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:35:28.167533 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:35:28.167533 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:35:28.167533 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:35:28.167533 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:35:28.167533 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 10:35:28.167533 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 10:35:28.167533 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 10:35:28.168452 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 10:35:28.168452 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:35:28.168452 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:35:28.168452 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:35:28.168452 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:35:28.168452 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:35:28.168452 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:35:28.168452 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:35:28.168452 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application with Lock( Jul 27 10:35:28.168452 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:35:28.168452 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:35:28.168452 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:35:28.168452 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:35:28.168452 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:35:28.168452 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:35:28.168452 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:35:28.168452 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:35:28.169332 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 10:35:28.169332 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 10:35:28.169332 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 10:35:28.169332 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Jul 27 10:35:28.169332 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 410/830] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 10:35:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 10:35:28.172517 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a278fd-ccdf-4f0a-a6c9-bf618906020d None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:28.175507 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef59f05b-d223-45bb-b8b2-6cf9f95dd7a6 None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:28.278853 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a278fd-ccdf-4f0a-a6c9-bf618906020d None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:28.303321 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef59f05b-d223-45bb-b8b2-6cf9f95dd7a6 None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:28.314703 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31a47ba-3fd1-46f7-9235-a478ee65d15c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:28.315360 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31a47ba-3fd1-46f7-9235-a478ee65d15c None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:28.315645 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31a47ba-3fd1-46f7-9235-a478ee65d15c None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:28.315924 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31a47ba-3fd1-46f7-9235-a478ee65d15c None admin] PATH_INFO: `/v3/roles/service` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:28.321271 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:28.327841 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31a47ba-3fd1-46f7-9235-a478ee65d15c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 10:35:28.332819 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application [None req-e31a47ba-3fd1-46f7-9235-a478ee65d15c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 27 10:35:28.332819 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:35:28.332819 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:35:28.332819 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:35:28.332819 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:35:28.332819 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:35:28.332819 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:35:28.332819 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:35:28.332819 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:35:28.332819 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:35:28.332819 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:35:28.332819 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:35:28.332819 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 10:35:28.332819 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 10:35:28.332819 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 10:35:28.332819 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 10:35:28.333726 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 10:35:28.333726 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 10:35:28.333726 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 10:35:28.333726 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 10:35:28.333726 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:35:28.333726 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:35:28.333726 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:35:28.333726 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:35:28.333726 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:35:28.333726 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:35:28.333726 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:35:28.333726 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application with Lock( Jul 27 10:35:28.333726 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:35:28.333726 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:35:28.333726 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:35:28.333726 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:35:28.335225 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:35:28.335225 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:35:28.335225 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:35:28.335225 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:35:28.335225 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 10:35:28.335225 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 10:35:28.335225 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 10:35:28.335225 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 10:35:28.335225 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 10:35:28.335225 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 10:35:28.335225 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 27 10:35:28.335225 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Jul 27 10:35:28.338845 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 421/831] 10.222.0.36 () {56 vars in 1102 bytes} [Wed Jul 27 10:35:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 10:35:28.340553 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31a47ba-3fd1-46f7-9235-a478ee65d15c None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:28.352901 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e144bed5-4b72-4c4a-960d-1e81d076b6e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:28.354324 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e144bed5-4b72-4c4a-960d-1e81d076b6e6 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:28.355092 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e144bed5-4b72-4c4a-960d-1e81d076b6e6 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:28.355585 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e144bed5-4b72-4c4a-960d-1e81d076b6e6 None admin] PATH_INFO: `/v3/domains` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:28.356726 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e144bed5-4b72-4c4a-960d-1e81d076b6e6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86828) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:35:28.357754 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e144bed5-4b72-4c4a-960d-1e81d076b6e6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:28.359875 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e144bed5-4b72-4c4a-960d-1e81d076b6e6 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:28.377787 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:28.378750 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 411/832] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 10:35:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:35:28.383249 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e144bed5-4b72-4c4a-960d-1e81d076b6e6 None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:28.455748 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31a47ba-3fd1-46f7-9235-a478ee65d15c None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:28.495192 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ccc0a1-b11b-4c29-b8f2-cf325ecd7532 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:28.496128 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ccc0a1-b11b-4c29-b8f2-cf325ecd7532 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:28.496412 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ccc0a1-b11b-4c29-b8f2-cf325ecd7532 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:28.496691 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ccc0a1-b11b-4c29-b8f2-cf325ecd7532 None admin] PATH_INFO: `/v3/roles` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:28.497420 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ccc0a1-b11b-4c29-b8f2-cf325ecd7532 None admin] RBAC: Adding query filter params (name=service) {{(pid=86827) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:35:28.497730 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ccc0a1-b11b-4c29-b8f2-cf325ecd7532 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:28.499581 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ccc0a1-b11b-4c29-b8f2-cf325ecd7532 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:28.510996 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 422/833] 10.222.0.36 () {56 vars in 1103 bytes} [Wed Jul 27 10:35:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 10:35:28.516216 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ccc0a1-b11b-4c29-b8f2-cf325ecd7532 None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:28.519419 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e144bed5-4b72-4c4a-960d-1e81d076b6e6 None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:28.557847 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87d890b-d7fb-4107-befe-3589c6f30cc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:28.558497 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87d890b-d7fb-4107-befe-3589c6f30cc5 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:28.558817 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87d890b-d7fb-4107-befe-3589c6f30cc5 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:28.559089 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87d890b-d7fb-4107-befe-3589c6f30cc5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:28.569401 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87d890b-d7fb-4107-befe-3589c6f30cc5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:28.571143 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87d890b-d7fb-4107-befe-3589c6f30cc5 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:28.583585 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application [None req-e87d890b-d7fb-4107-befe-3589c6f30cc5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 10:35:28.583585 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:35:28.583585 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:35:28.583585 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:35:28.583585 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:35:28.583585 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:35:28.583585 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:35:28.583585 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:35:28.583585 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:35:28.583585 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:35:28.583585 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:35:28.583585 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:35:28.583585 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 10:35:28.583585 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 10:35:28.583585 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 10:35:28.584522 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 10:35:28.584522 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:35:28.584522 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:35:28.584522 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:35:28.584522 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:35:28.584522 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:35:28.584522 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:35:28.584522 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:35:28.584522 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application with Lock( Jul 27 10:35:28.584522 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:35:28.584522 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:35:28.584522 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:35:28.584522 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:35:28.584522 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:35:28.584522 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:35:28.584522 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:35:28.584522 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:35:28.585431 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 10:35:28.585431 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 10:35:28.585431 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 10:35:28.585431 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 10:35:28.585431 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 10:35:28.585431 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 10:35:28.585431 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 10:35:28.585431 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Jul 27 10:35:28.585431 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 412/834] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Jul 27 10:35:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 10:35:28.592153 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87d890b-d7fb-4107-befe-3589c6f30cc5 None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:28.634086 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:28.636180 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:28.642268 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ccc0a1-b11b-4c29-b8f2-cf325ecd7532 None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:28.683129 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2abc0cb-455e-4bd7-8ca8-4b2c07a6671b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:28.683767 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2abc0cb-455e-4bd7-8ca8-4b2c07a6671b None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:28.684369 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2abc0cb-455e-4bd7-8ca8-4b2c07a6671b None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:28.684369 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2abc0cb-455e-4bd7-8ca8-4b2c07a6671b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:28.695256 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2abc0cb-455e-4bd7-8ca8-4b2c07a6671b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:28.696685 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2abc0cb-455e-4bd7-8ca8-4b2c07a6671b None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:28.711469 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application [None req-f2abc0cb-455e-4bd7-8ca8-4b2c07a6671b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 10:35:28.711469 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:35:28.711469 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:35:28.711469 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:35:28.711469 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:35:28.711469 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:35:28.711469 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:35:28.711469 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:35:28.711469 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:35:28.711469 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:35:28.711469 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:35:28.711469 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:35:28.711469 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 10:35:28.711469 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 10:35:28.711469 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 10:35:28.712342 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 10:35:28.712342 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:35:28.712342 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:35:28.712342 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:35:28.712342 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:35:28.712342 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:35:28.712342 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:35:28.712342 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:35:28.712342 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application with Lock( Jul 27 10:35:28.712342 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:35:28.712342 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:35:28.712342 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:35:28.712342 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:35:28.712342 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:35:28.712342 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:35:28.712342 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:35:28.712342 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:35:28.713248 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 10:35:28.713248 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 10:35:28.713248 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 10:35:28.713248 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Jul 27 10:35:28.713248 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 423/835] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 10:35:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 10:35:28.714098 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ce0d96-8306-4c77-be8d-32cd6829d461 None None] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:28.714385 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ce0d96-8306-4c77-be8d-32cd6829d461 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:28.714696 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ce0d96-8306-4c77-be8d-32cd6829d461 None None] PATH_INFO: `/` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:28.716063 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 424/836] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 10:35:28 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 10:35:28.719296 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87d890b-d7fb-4107-befe-3589c6f30cc5 None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:28.719705 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ce0d96-8306-4c77-be8d-32cd6829d461 None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:28.755789 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77e0b6d-e229-4d5b-8dd8-f884f97479c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:28.756438 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77e0b6d-e229-4d5b-8dd8-f884f97479c2 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:28.756719 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77e0b6d-e229-4d5b-8dd8-f884f97479c2 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:28.756990 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77e0b6d-e229-4d5b-8dd8-f884f97479c2 None admin] PATH_INFO: `/v3/projects` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:28.757739 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77e0b6d-e229-4d5b-8dd8-f884f97479c2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86828) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:35:28.758039 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77e0b6d-e229-4d5b-8dd8-f884f97479c2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:28.759937 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77e0b6d-e229-4d5b-8dd8-f884f97479c2 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:28.777619 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 413/837] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Jul 27 10:35:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:35:28.777619 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2a87bb-9a66-4817-842e-d51b4bc12ceb None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:28.777619 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2a87bb-9a66-4817-842e-d51b4bc12ceb None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:28.777619 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2a87bb-9a66-4817-842e-d51b4bc12ceb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:28.793676 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-0e2a87bb-9a66-4817-842e-d51b4bc12ceb None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:35:28.794813 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-0e2a87bb-9a66-4817-842e-d51b4bc12ceb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:35:28.834886 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ce0d96-8306-4c77-be8d-32cd6829d461 None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:28.867243 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 414/838] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 10:35:28 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 10:35:28.867729 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da351bc-43de-4291-9295-36de87e3b9b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:28.868526 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da351bc-43de-4291-9295-36de87e3b9b3 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:28.868809 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da351bc-43de-4291-9295-36de87e3b9b3 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:28.869083 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da351bc-43de-4291-9295-36de87e3b9b3 None admin] PATH_INFO: `/v3/domains` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:28.869303 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2a87bb-9a66-4817-842e-d51b4bc12ceb None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:28.869772 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da351bc-43de-4291-9295-36de87e3b9b3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86827) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:35:28.870101 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da351bc-43de-4291-9295-36de87e3b9b3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:28.871454 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da351bc-43de-4291-9295-36de87e3b9b3 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:28.894419 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 425/839] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 10:35:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:35:28.895485 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da351bc-43de-4291-9295-36de87e3b9b3 None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:28.902667 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:28.907690 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:28.990924 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2a87bb-9a66-4817-842e-d51b4bc12ceb None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:29.007667 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da351bc-43de-4291-9295-36de87e3b9b3 None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:29.027493 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17370343-4d0d-484e-8561-0272218875ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:29.028123 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17370343-4d0d-484e-8561-0272218875ad None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:29.028409 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17370343-4d0d-484e-8561-0272218875ad None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:29.028691 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17370343-4d0d-484e-8561-0272218875ad None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:29.029487 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17370343-4d0d-484e-8561-0272218875ad None admin] RBAC: Adding query filter params (role.id=21156560eb6b436bb567a40b4976f3f4, scope.project.id=5f296d8e3b8f4001909addc49d8dac01, user.id=e059d8b6c28c4fcba925d05d7c98e4ac) {{(pid=86828) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:35:29.029808 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17370343-4d0d-484e-8561-0272218875ad None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:29.031268 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17370343-4d0d-484e-8561-0272218875ad None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:29.038466 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 415/840] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Jul 27 10:35:28 2022] GET /identity/v3/role_assignments?user.id=e059d8b6c28c4fcba925d05d7c98e4ac&scope.project.id=5f296d8e3b8f4001909addc49d8dac01&role.id=21156560eb6b436bb567a40b4976f3f4 => generated 608 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:35:29.039299 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17370343-4d0d-484e-8561-0272218875ad None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:29.054264 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb81d98-15fa-4182-9fdb-db3e53430193 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:29.055784 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb81d98-15fa-4182-9fdb-db3e53430193 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:29.056215 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb81d98-15fa-4182-9fdb-db3e53430193 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:29.056636 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb81d98-15fa-4182-9fdb-db3e53430193 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:29.057468 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb81d98-15fa-4182-9fdb-db3e53430193 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:29.059236 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb81d98-15fa-4182-9fdb-db3e53430193 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:29.070365 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application [None req-2bb81d98-15fa-4182-9fdb-db3e53430193 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 27 10:35:29.070365 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:35:29.070365 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 27 10:35:29.070365 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 27 10:35:29.070365 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 27 10:35:29.070365 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 27 10:35:29.070365 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 27 10:35:29.070365 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 10:35:29.070365 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 27 10:35:29.070365 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Jul 27 10:35:29.070365 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 10:35:29.070365 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Jul 27 10:35:29.070365 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:35:29.070365 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:35:29.070365 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:35:29.070365 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:35:29.070365 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:35:29.071649 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:35:29.071649 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:35:29.071649 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:35:29.071649 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:35:29.071649 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:35:29.071649 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:35:29.071649 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 27 10:35:29.071649 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 27 10:35:29.071649 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 27 10:35:29.071649 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 27 10:35:29.071649 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:35:29.071649 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:35:29.071649 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:35:29.071649 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:35:29.071649 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:35:29.071649 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:35:29.072682 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:35:29.072682 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application with Lock( Jul 27 10:35:29.072682 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:35:29.072682 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:35:29.072682 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:35:29.072682 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:35:29.072682 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:35:29.072682 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:35:29.072682 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:35:29.072682 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:35:29.072682 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 27 10:35:29.072682 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 10:35:29.072682 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 27 10:35:29.072682 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Jul 27 10:35:29.080574 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 426/841] 10.222.0.36 () {56 vars in 1114 bytes} [Wed Jul 27 10:35:28 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 10:35:29.082153 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb81d98-15fa-4182-9fdb-db3e53430193 None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:29.159043 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:29.184372 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:29.186152 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17370343-4d0d-484e-8561-0272218875ad None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:29.216653 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb81d98-15fa-4182-9fdb-db3e53430193 None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:29.238280 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6714afa0-a731-4372-a733-0e63f29b5716 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:29.238984 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6714afa0-a731-4372-a733-0e63f29b5716 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:29.239272 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6714afa0-a731-4372-a733-0e63f29b5716 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:29.239557 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6714afa0-a731-4372-a733-0e63f29b5716 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:29.254031 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6714afa0-a731-4372-a733-0e63f29b5716 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:29.256925 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6714afa0-a731-4372-a733-0e63f29b5716 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:29.260987 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410fd4ed-0e0b-4c48-943e-6da64376ec49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:29.261742 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410fd4ed-0e0b-4c48-943e-6da64376ec49 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:29.262139 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410fd4ed-0e0b-4c48-943e-6da64376ec49 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:29.262514 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410fd4ed-0e0b-4c48-943e-6da64376ec49 None admin] PATH_INFO: `/v3/services` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:29.263365 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410fd4ed-0e0b-4c48-943e-6da64376ec49 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=86827) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:35:29.264641 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410fd4ed-0e0b-4c48-943e-6da64376ec49 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:29.266192 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410fd4ed-0e0b-4c48-943e-6da64376ec49 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:29.272071 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 427/842] 10.222.0.36 () {56 vars in 1114 bytes} [Wed Jul 27 10:35:29 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:35:29.274516 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application [None req-6714afa0-a731-4372-a733-0e63f29b5716 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 27 10:35:29.274516 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:35:29.274516 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:35:29.274516 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:35:29.274516 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:35:29.274516 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:35:29.274516 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:35:29.274516 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:35:29.274516 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:35:29.274516 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:35:29.274516 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:35:29.274516 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:35:29.274516 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 10:35:29.274516 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 10:35:29.274516 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 10:35:29.274516 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 10:35:29.275466 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:35:29.275466 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:35:29.275466 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 10:35:29.275466 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 10:35:29.275466 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 10:35:29.275466 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 10:35:29.275466 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:35:29.275466 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:35:29.275466 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:35:29.275466 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:35:29.275466 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:35:29.275466 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application with Lock( Jul 27 10:35:29.275466 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:35:29.275466 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:35:29.275466 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:35:29.275466 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:35:29.275466 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:35:29.275466 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:35:29.276394 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:35:29.276394 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:35:29.276394 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 10:35:29.276394 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 10:35:29.276394 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 10:35:29.276394 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 10:35:29.276394 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 10:35:29.276394 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 10:35:29.276394 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 27 10:35:29.276394 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Jul 27 10:35:29.276921 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 416/843] 10.222.0.36 () {56 vars in 1117 bytes} [Wed Jul 27 10:35:29 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 10:35:29.277726 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410fd4ed-0e0b-4c48-943e-6da64376ec49 None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:29.282327 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6714afa0-a731-4372-a733-0e63f29b5716 None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:29.403036 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410fd4ed-0e0b-4c48-943e-6da64376ec49 None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:29.408494 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6714afa0-a731-4372-a733-0e63f29b5716 None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:29.441445 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3a6490-889c-4f01-83f7-f31eed2bdb96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:29.441987 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3a6490-889c-4f01-83f7-f31eed2bdb96 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:29.442266 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3a6490-889c-4f01-83f7-f31eed2bdb96 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:29.442536 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3a6490-889c-4f01-83f7-f31eed2bdb96 None admin] PATH_INFO: `/v3/services` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:29.443243 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3a6490-889c-4f01-83f7-f31eed2bdb96 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=86827) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:35:29.443582 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3a6490-889c-4f01-83f7-f31eed2bdb96 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:29.444410 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722aaf9c-a720-47d0-aa6c-bb5983b0cc84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:29.445039 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3a6490-889c-4f01-83f7-f31eed2bdb96 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:29.445212 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722aaf9c-a720-47d0-aa6c-bb5983b0cc84 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:29.445589 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722aaf9c-a720-47d0-aa6c-bb5983b0cc84 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:29.445961 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722aaf9c-a720-47d0-aa6c-bb5983b0cc84 None admin] PATH_INFO: `/v3/users` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:29.446901 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722aaf9c-a720-47d0-aa6c-bb5983b0cc84 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=86828) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:35:29.447318 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722aaf9c-a720-47d0-aa6c-bb5983b0cc84 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:29.448816 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722aaf9c-a720-47d0-aa6c-bb5983b0cc84 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:29.458630 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 428/844] 10.222.0.36 () {56 vars in 1114 bytes} [Wed Jul 27 10:35:29 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 10:35:29.458630 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a8940a-c405-4bba-afda-e5cc7f1c228f None None] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:29.458630 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a8940a-c405-4bba-afda-e5cc7f1c228f None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:29.458630 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a8940a-c405-4bba-afda-e5cc7f1c228f None None] PATH_INFO: `/` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:29.458630 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 429/845] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 10:35:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 10:35:29.459969 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25175168-682a-492a-b0f9-f8e387b8af5f None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:29.460257 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25175168-682a-492a-b0f9-f8e387b8af5f None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:29.460535 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25175168-682a-492a-b0f9-f8e387b8af5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:29.467601 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:29.479608 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 417/846] 10.222.0.36 () {56 vars in 1149 bytes} [Wed Jul 27 10:35:29 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:35:29.486356 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722aaf9c-a720-47d0-aa6c-bb5983b0cc84 None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:29.487963 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-25175168-682a-492a-b0f9-f8e387b8af5f None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:35:29.489364 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-25175168-682a-492a-b0f9-f8e387b8af5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:35:29.490803 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:29.586705 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 430/847] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 10:35:29 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 27 10:35:29.608853 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25175168-682a-492a-b0f9-f8e387b8af5f None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:29.628248 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722aaf9c-a720-47d0-aa6c-bb5983b0cc84 None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:29.673170 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58385002-cc42-453e-9ca7-a58f0408a30a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:29.673844 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58385002-cc42-453e-9ca7-a58f0408a30a None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:29.674128 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58385002-cc42-453e-9ca7-a58f0408a30a None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:29.674537 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58385002-cc42-453e-9ca7-a58f0408a30a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:29.739135 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25175168-682a-492a-b0f9-f8e387b8af5f None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:29.753024 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58385002-cc42-453e-9ca7-a58f0408a30a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:29.754854 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58385002-cc42-453e-9ca7-a58f0408a30a None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:29.769392 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:29.774958 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:29.775651 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edabec40-6ae6-4f0f-a1a0-77ffd393fcb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:29.776479 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edabec40-6ae6-4f0f-a1a0-77ffd393fcb7 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:29.777090 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edabec40-6ae6-4f0f-a1a0-77ffd393fcb7 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:29.777407 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edabec40-6ae6-4f0f-a1a0-77ffd393fcb7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:29.789163 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edabec40-6ae6-4f0f-a1a0-77ffd393fcb7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 10:35:29.794368 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application [None req-edabec40-6ae6-4f0f-a1a0-77ffd393fcb7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 10:35:29.794368 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:35:29.794368 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:35:29.794368 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:35:29.794368 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:35:29.794368 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:35:29.794368 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:35:29.794368 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:35:29.794368 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:35:29.794368 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:35:29.794368 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:35:29.794368 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:35:29.794368 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 10:35:29.794368 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 10:35:29.794368 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 10:35:29.794368 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 10:35:29.795545 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 10:35:29.795545 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 10:35:29.795545 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 10:35:29.795545 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 10:35:29.795545 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:35:29.795545 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:35:29.795545 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:35:29.795545 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:35:29.795545 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:35:29.795545 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:35:29.795545 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:35:29.795545 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application with Lock( Jul 27 10:35:29.795545 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:35:29.795545 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:35:29.795545 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:35:29.795545 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:35:29.796489 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:35:29.796489 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:35:29.796489 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:35:29.796489 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:35:29.796489 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 10:35:29.796489 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 10:35:29.796489 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 10:35:29.796489 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 10:35:29.796489 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 10:35:29.796489 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 10:35:29.796489 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 10:35:29.796489 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Jul 27 10:35:29.797567 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 431/848] 10.222.0.36 () {56 vars in 1096 bytes} [Wed Jul 27 10:35:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 10:35:29.803917 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edabec40-6ae6-4f0f-a1a0-77ffd393fcb7 None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:29.893430 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 418/849] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jul 27 10:35:29 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 27 10:35:29.895264 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58385002-cc42-453e-9ca7-a58f0408a30a None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:29.936329 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edabec40-6ae6-4f0f-a1a0-77ffd393fcb7 None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:29.980934 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2482689-3795-441b-8e44-fb805b41cc6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:29.981551 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2482689-3795-441b-8e44-fb805b41cc6f None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:29.981827 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2482689-3795-441b-8e44-fb805b41cc6f None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:29.982096 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2482689-3795-441b-8e44-fb805b41cc6f None admin] PATH_INFO: `/v3/roles` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:29.982836 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2482689-3795-441b-8e44-fb805b41cc6f None admin] RBAC: Adding query filter params (name=admin) {{(pid=86827) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:35:29.983140 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2482689-3795-441b-8e44-fb805b41cc6f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:29.984507 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2482689-3795-441b-8e44-fb805b41cc6f None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:29.998163 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 432/850] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jul 27 10:35:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:35:30.000582 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2482689-3795-441b-8e44-fb805b41cc6f None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:30.031245 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58385002-cc42-453e-9ca7-a58f0408a30a None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:30.034514 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:30.075605 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6a7ca8-c6da-4854-a273-f9660d754333 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:30.076254 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6a7ca8-c6da-4854-a273-f9660d754333 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:30.076529 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6a7ca8-c6da-4854-a273-f9660d754333 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:30.076800 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6a7ca8-c6da-4854-a273-f9660d754333 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:30.080833 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:30.089862 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6a7ca8-c6da-4854-a273-f9660d754333 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:30.091509 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6a7ca8-c6da-4854-a273-f9660d754333 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:30.110530 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application [None req-cf6a7ca8-c6da-4854-a273-f9660d754333 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 10:35:30.110530 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:35:30.110530 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:35:30.110530 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:35:30.110530 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:35:30.110530 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:35:30.110530 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:35:30.110530 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:35:30.110530 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:35:30.110530 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:35:30.110530 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:35:30.110530 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:35:30.110530 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 10:35:30.110530 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 10:35:30.110530 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 10:35:30.111501 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 10:35:30.111501 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:35:30.111501 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:35:30.111501 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:35:30.111501 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:35:30.111501 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:35:30.111501 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:35:30.111501 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:35:30.111501 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application with Lock( Jul 27 10:35:30.111501 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:35:30.111501 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:35:30.111501 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:35:30.111501 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:35:30.111501 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:35:30.111501 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:35:30.111501 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:35:30.111501 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:35:30.112394 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 10:35:30.112394 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 10:35:30.112394 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 10:35:30.112394 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Jul 27 10:35:30.114738 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 419/851] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 10:35:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 10:35:30.118465 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6a7ca8-c6da-4854-a273-f9660d754333 None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:30.144875 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2482689-3795-441b-8e44-fb805b41cc6f None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:30.190987 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02899a1c-675f-45ad-adfc-07a7bf5e3de3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:30.192681 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02899a1c-675f-45ad-adfc-07a7bf5e3de3 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:30.193180 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02899a1c-675f-45ad-adfc-07a7bf5e3de3 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:30.193936 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02899a1c-675f-45ad-adfc-07a7bf5e3de3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:30.211011 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02899a1c-675f-45ad-adfc-07a7bf5e3de3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:30.213299 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02899a1c-675f-45ad-adfc-07a7bf5e3de3 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:30.230311 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application [None req-02899a1c-675f-45ad-adfc-07a7bf5e3de3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 10:35:30.230311 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:35:30.230311 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:35:30.230311 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:35:30.230311 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:35:30.230311 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:35:30.230311 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:35:30.230311 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:35:30.230311 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:35:30.230311 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:35:30.230311 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:35:30.230311 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:35:30.230311 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 10:35:30.230311 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 10:35:30.230311 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 10:35:30.231281 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 10:35:30.231281 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:35:30.231281 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:35:30.231281 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:35:30.231281 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:35:30.231281 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:35:30.231281 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:35:30.231281 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:35:30.231281 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application with Lock( Jul 27 10:35:30.231281 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:35:30.231281 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:35:30.231281 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:35:30.231281 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:35:30.231281 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:35:30.231281 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:35:30.231281 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:35:30.231281 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:35:30.232364 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 10:35:30.232364 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 10:35:30.232364 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 10:35:30.232364 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Jul 27 10:35:30.232364 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 433/852] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 10:35:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 10:35:30.239558 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02899a1c-675f-45ad-adfc-07a7bf5e3de3 None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:30.239994 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6a7ca8-c6da-4854-a273-f9660d754333 None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:30.284895 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8630081d-5927-44b5-8405-3621c1d363be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:30.285511 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8630081d-5927-44b5-8405-3621c1d363be None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:30.285832 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8630081d-5927-44b5-8405-3621c1d363be None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:30.286234 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8630081d-5927-44b5-8405-3621c1d363be None admin] PATH_INFO: `/v3/domains` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:30.287000 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8630081d-5927-44b5-8405-3621c1d363be None admin] RBAC: Adding query filter params (name=Default) {{(pid=86828) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:35:30.288722 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8630081d-5927-44b5-8405-3621c1d363be None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:30.290109 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8630081d-5927-44b5-8405-3621c1d363be None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:30.309449 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 420/853] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 10:35:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 10:35:30.310325 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8630081d-5927-44b5-8405-3621c1d363be None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:30.333913 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:30.382842 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02899a1c-675f-45ad-adfc-07a7bf5e3de3 None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:30.392785 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:30.420868 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8630081d-5927-44b5-8405-3621c1d363be None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:30.433262 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd835275-c2e5-4f7d-9d03-4fcb86742c9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:30.434409 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd835275-c2e5-4f7d-9d03-4fcb86742c9f None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:30.434895 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd835275-c2e5-4f7d-9d03-4fcb86742c9f None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:30.435587 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd835275-c2e5-4f7d-9d03-4fcb86742c9f None admin] PATH_INFO: `/v3/domains` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:30.436932 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd835275-c2e5-4f7d-9d03-4fcb86742c9f None admin] RBAC: Adding query filter params (name=Default) {{(pid=86827) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:35:30.437698 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd835275-c2e5-4f7d-9d03-4fcb86742c9f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:30.439571 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd835275-c2e5-4f7d-9d03-4fcb86742c9f None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:30.456902 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 434/854] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 10:35:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:35:30.458072 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd835275-c2e5-4f7d-9d03-4fcb86742c9f None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:30.464813 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c580e90f-9834-4345-985b-502992ed33f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:30.465738 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c580e90f-9834-4345-985b-502992ed33f6 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:30.466270 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c580e90f-9834-4345-985b-502992ed33f6 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:30.466764 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c580e90f-9834-4345-985b-502992ed33f6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:30.479896 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c580e90f-9834-4345-985b-502992ed33f6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:30.484159 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c580e90f-9834-4345-985b-502992ed33f6 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:30.501131 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application [None req-c580e90f-9834-4345-985b-502992ed33f6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 10:35:30.501131 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:35:30.501131 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:35:30.501131 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:35:30.501131 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:35:30.501131 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:35:30.501131 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:35:30.501131 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:35:30.501131 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:35:30.501131 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:35:30.501131 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:35:30.501131 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:35:30.501131 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 10:35:30.501131 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 10:35:30.501131 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 10:35:30.502233 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 10:35:30.502233 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:35:30.502233 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:35:30.502233 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:35:30.502233 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:35:30.502233 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:35:30.502233 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:35:30.502233 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:35:30.502233 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application with Lock( Jul 27 10:35:30.502233 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:35:30.502233 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:35:30.502233 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:35:30.502233 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:35:30.502233 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:35:30.502233 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:35:30.502233 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:35:30.502233 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:35:30.503363 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 10:35:30.503363 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 10:35:30.503363 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 10:35:30.503363 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 10:35:30.503363 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 10:35:30.503363 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 10:35:30.503363 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 10:35:30.503363 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Jul 27 10:35:30.505423 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 421/855] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Jul 27 10:35:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 10:35:30.511256 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c580e90f-9834-4345-985b-502992ed33f6 None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:30.575776 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd835275-c2e5-4f7d-9d03-4fcb86742c9f None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:30.617588 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843ecea5-2cc1-48c0-a833-00e301edda89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:30.617588 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843ecea5-2cc1-48c0-a833-00e301edda89 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:30.617865 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843ecea5-2cc1-48c0-a833-00e301edda89 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:30.618121 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843ecea5-2cc1-48c0-a833-00e301edda89 None admin] PATH_INFO: `/v3/users/swift` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:30.626087 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843ecea5-2cc1-48c0-a833-00e301edda89 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:30.627562 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843ecea5-2cc1-48c0-a833-00e301edda89 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:30.641304 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application [None req-843ecea5-2cc1-48c0-a833-00e301edda89 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 27 10:35:30.641304 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:35:30.641304 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:35:30.641304 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:35:30.641304 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:35:30.641304 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:35:30.641304 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:35:30.641304 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:35:30.641304 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:35:30.641304 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:35:30.641304 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:35:30.641304 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:35:30.641304 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 10:35:30.641304 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 10:35:30.641304 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 10:35:30.641304 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 10:35:30.642354 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:35:30.642354 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:35:30.642354 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 10:35:30.642354 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 10:35:30.642354 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 10:35:30.642354 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 10:35:30.642354 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:35:30.642354 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:35:30.642354 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:35:30.642354 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:35:30.642354 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:35:30.642354 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application with Lock( Jul 27 10:35:30.642354 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:35:30.642354 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:35:30.642354 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:35:30.642354 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:35:30.642354 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:35:30.642354 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:35:30.643339 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:35:30.643339 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:35:30.643339 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 10:35:30.643339 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 10:35:30.643339 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 10:35:30.643339 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 10:35:30.643339 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 10:35:30.643339 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 10:35:30.643339 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 27 10:35:30.643339 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Jul 27 10:35:30.647999 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 435/856] 10.222.0.36 () {56 vars in 1096 bytes} [Wed Jul 27 10:35:30 2022] GET /identity/v3/users/swift => generated 83 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 10:35:30.649595 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843ecea5-2cc1-48c0-a833-00e301edda89 None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:30.657871 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c580e90f-9834-4345-985b-502992ed33f6 None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:30.674999 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:30.677903 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:30.699732 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6629c5e4-6384-4a6c-a6b8-07763a2afc57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:30.700724 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6629c5e4-6384-4a6c-a6b8-07763a2afc57 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:30.701207 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6629c5e4-6384-4a6c-a6b8-07763a2afc57 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:30.701693 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6629c5e4-6384-4a6c-a6b8-07763a2afc57 None admin] PATH_INFO: `/v3/projects` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:30.702801 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6629c5e4-6384-4a6c-a6b8-07763a2afc57 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86828) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:35:30.703375 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6629c5e4-6384-4a6c-a6b8-07763a2afc57 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:30.705016 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6629c5e4-6384-4a6c-a6b8-07763a2afc57 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:30.718657 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 422/857] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Jul 27 10:35:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:35:30.724410 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6629c5e4-6384-4a6c-a6b8-07763a2afc57 None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:30.773485 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843ecea5-2cc1-48c0-a833-00e301edda89 None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:30.816679 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d85da8b-1f5f-4569-b913-88de59264b9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:30.817305 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d85da8b-1f5f-4569-b913-88de59264b9e None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:30.817586 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d85da8b-1f5f-4569-b913-88de59264b9e None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:30.817863 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d85da8b-1f5f-4569-b913-88de59264b9e None admin] PATH_INFO: `/v3/users` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:30.819225 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d85da8b-1f5f-4569-b913-88de59264b9e None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=86827) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:35:30.819668 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d85da8b-1f5f-4569-b913-88de59264b9e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:30.821094 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d85da8b-1f5f-4569-b913-88de59264b9e None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:30.847968 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 436/858] 10.222.0.36 () {56 vars in 1135 bytes} [Wed Jul 27 10:35:30 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 10:35:30.849561 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d85da8b-1f5f-4569-b913-88de59264b9e None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:30.855506 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6629c5e4-6384-4a6c-a6b8-07763a2afc57 None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:30.892241 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7df19d-0912-4344-8454-9182b54afac7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:30.892864 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7df19d-0912-4344-8454-9182b54afac7 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:30.893196 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7df19d-0912-4344-8454-9182b54afac7 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:30.893477 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7df19d-0912-4344-8454-9182b54afac7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:30.894269 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7df19d-0912-4344-8454-9182b54afac7 None admin] RBAC: Adding query filter params (role.id=9e0038c6bbd34556af3f84f9e2a7c2c5, scope.project.id=5f296d8e3b8f4001909addc49d8dac01, user.id=3d9fed24b4ba44b0b5961a603c4362c4) {{(pid=86828) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:35:30.894577 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7df19d-0912-4344-8454-9182b54afac7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:30.896097 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7df19d-0912-4344-8454-9182b54afac7 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:30.900082 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:30.912141 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 423/859] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Jul 27 10:35:30 2022] GET /identity/v3/role_assignments?user.id=3d9fed24b4ba44b0b5961a603c4362c4&scope.project.id=5f296d8e3b8f4001909addc49d8dac01&role.id=9e0038c6bbd34556af3f84f9e2a7c2c5 => generated 608 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:35:30.976362 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:30.995116 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d85da8b-1f5f-4569-b913-88de59264b9e None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:31.033497 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acc8e11-311e-4e19-b10d-5669a49b919e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:31.034143 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acc8e11-311e-4e19-b10d-5669a49b919e None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:31.034424 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acc8e11-311e-4e19-b10d-5669a49b919e None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:31.034724 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acc8e11-311e-4e19-b10d-5669a49b919e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:31.109428 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6acc8e11-311e-4e19-b10d-5669a49b919e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:31.110845 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6acc8e11-311e-4e19-b10d-5669a49b919e None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:31.185885 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077770e6-0ee4-4750-b69e-f3bfbd5e429f None None] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:31.186393 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077770e6-0ee4-4750-b69e-f3bfbd5e429f None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:31.186855 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077770e6-0ee4-4750-b69e-f3bfbd5e429f None None] PATH_INFO: `/` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:31.189273 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 424/860] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 10:35:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 10:35:31.194253 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6119f8-d13d-441c-8481-200cc72df73e None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:31.194540 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6119f8-d13d-441c-8481-200cc72df73e None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:31.195211 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6119f8-d13d-441c-8481-200cc72df73e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:31.202774 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:31.217153 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-eb6119f8-d13d-441c-8481-200cc72df73e None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:35:31.218676 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-eb6119f8-d13d-441c-8481-200cc72df73e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:35:31.229146 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 437/861] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jul 27 10:35:30 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 27 10:35:31.234631 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acc8e11-311e-4e19-b10d-5669a49b919e None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:31.293168 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 425/862] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 10:35:31 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 10:35:31.315738 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6119f8-d13d-441c-8481-200cc72df73e None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:31.355662 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acc8e11-311e-4e19-b10d-5669a49b919e None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:31.401112 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61007ca0-de87-4657-9d44-2fac1a5c58b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:31.401688 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61007ca0-de87-4657-9d44-2fac1a5c58b9 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:31.401968 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61007ca0-de87-4657-9d44-2fac1a5c58b9 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:31.402241 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61007ca0-de87-4657-9d44-2fac1a5c58b9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:31.416277 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61007ca0-de87-4657-9d44-2fac1a5c58b9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:31.417818 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61007ca0-de87-4657-9d44-2fac1a5c58b9 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:31.438616 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6119f8-d13d-441c-8481-200cc72df73e None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:31.448643 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application [None req-61007ca0-de87-4657-9d44-2fac1a5c58b9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 10:35:31.448643 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:35:31.448643 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:35:31.448643 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:35:31.448643 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:35:31.448643 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:35:31.448643 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:35:31.448643 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:35:31.448643 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:35:31.448643 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:35:31.448643 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:35:31.448643 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:35:31.448643 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 10:35:31.448643 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 10:35:31.448643 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 10:35:31.449558 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 10:35:31.449558 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:35:31.449558 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:35:31.449558 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:35:31.449558 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:35:31.449558 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:35:31.449558 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:35:31.449558 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:35:31.449558 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application with Lock( Jul 27 10:35:31.449558 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:35:31.449558 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:35:31.449558 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:35:31.449558 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:35:31.449558 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:35:31.449558 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:35:31.449558 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:35:31.449558 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:35:31.450420 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 10:35:31.450420 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 10:35:31.450420 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 10:35:31.450420 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Jul 27 10:35:31.457063 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 438/863] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 10:35:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 10:35:31.457743 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61007ca0-de87-4657-9d44-2fac1a5c58b9 None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:31.477611 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:31.485032 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57309e1f-d499-4958-a2d4-60a14da395ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:31.485745 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57309e1f-d499-4958-a2d4-60a14da395ac None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:31.486032 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57309e1f-d499-4958-a2d4-60a14da395ac None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:31.486299 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57309e1f-d499-4958-a2d4-60a14da395ac None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:31.486970 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57309e1f-d499-4958-a2d4-60a14da395ac None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:31.488408 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57309e1f-d499-4958-a2d4-60a14da395ac None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:31.498805 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application [None req-57309e1f-d499-4958-a2d4-60a14da395ac None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 27 10:35:31.498805 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:35:31.498805 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 27 10:35:31.498805 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 27 10:35:31.498805 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 27 10:35:31.498805 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 27 10:35:31.498805 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 27 10:35:31.498805 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 10:35:31.498805 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 27 10:35:31.498805 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Jul 27 10:35:31.498805 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 10:35:31.498805 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Jul 27 10:35:31.498805 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:35:31.498805 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:35:31.498805 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:35:31.498805 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:35:31.499771 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:35:31.499771 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:35:31.499771 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:35:31.499771 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:35:31.499771 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:35:31.499771 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:35:31.499771 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:35:31.499771 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 27 10:35:31.499771 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 27 10:35:31.499771 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 27 10:35:31.499771 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 27 10:35:31.499771 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:35:31.499771 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:35:31.499771 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:35:31.499771 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:35:31.499771 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:35:31.500592 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:35:31.500592 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:35:31.500592 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application with Lock( Jul 27 10:35:31.500592 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:35:31.500592 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:35:31.500592 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:35:31.500592 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:35:31.500592 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:35:31.500592 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:35:31.500592 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:35:31.500592 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:35:31.500592 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 27 10:35:31.500592 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 10:35:31.500592 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 27 10:35:31.500592 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Jul 27 10:35:31.505162 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 426/864] 10.222.0.36 () {56 vars in 1132 bytes} [Wed Jul 27 10:35:31 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 10:35:31.506265 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57309e1f-d499-4958-a2d4-60a14da395ac None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:31.511825 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:31.596127 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61007ca0-de87-4657-9d44-2fac1a5c58b9 None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:31.621453 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57309e1f-d499-4958-a2d4-60a14da395ac None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:31.637342 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b4c0d9-e12e-48d1-8a58-566366827de4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:31.637968 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b4c0d9-e12e-48d1-8a58-566366827de4 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:31.638258 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b4c0d9-e12e-48d1-8a58-566366827de4 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:31.638555 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b4c0d9-e12e-48d1-8a58-566366827de4 None admin] PATH_INFO: `/v3/domains` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:31.639259 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b4c0d9-e12e-48d1-8a58-566366827de4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86827) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:35:31.639568 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b4c0d9-e12e-48d1-8a58-566366827de4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:31.641117 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b4c0d9-e12e-48d1-8a58-566366827de4 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:31.658023 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 439/865] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 10:35:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 10:35:31.659330 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad0e1e2-4361-4c9c-86c2-1c30515bc5c0 None None] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:31.659620 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad0e1e2-4361-4c9c-86c2-1c30515bc5c0 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:31.659896 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad0e1e2-4361-4c9c-86c2-1c30515bc5c0 None None] PATH_INFO: `/` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:31.660117 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98652f39-9f3e-4c56-ba22-21d162f79360 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:31.661502 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98652f39-9f3e-4c56-ba22-21d162f79360 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:31.662054 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98652f39-9f3e-4c56-ba22-21d162f79360 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:31.663108 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98652f39-9f3e-4c56-ba22-21d162f79360 None admin] PATH_INFO: `/v3/services` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:31.663909 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98652f39-9f3e-4c56-ba22-21d162f79360 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=86828) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:35:31.664220 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98652f39-9f3e-4c56-ba22-21d162f79360 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:31.665613 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98652f39-9f3e-4c56-ba22-21d162f79360 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:31.665801 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 440/866] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 10:35:31 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 10:35:31.669983 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad0e1e2-4361-4c9c-86c2-1c30515bc5c0 None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:31.678927 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 427/867] 10.222.0.36 () {56 vars in 1126 bytes} [Wed Jul 27 10:35:31 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 10:35:31.678927 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ad6e0e-d834-4e7b-bce9-cfdff696509b None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:31.678927 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ad6e0e-d834-4e7b-bce9-cfdff696509b None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:31.678927 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ad6e0e-d834-4e7b-bce9-cfdff696509b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:31.700054 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-81ad6e0e-d834-4e7b-bce9-cfdff696509b None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:35:31.701259 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-81ad6e0e-d834-4e7b-bce9-cfdff696509b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:35:31.774938 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:31.786709 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:31.790275 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 428/868] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 10:35:31 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 10:35:31.795872 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad0e1e2-4361-4c9c-86c2-1c30515bc5c0 None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:31.799571 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ad6e0e-d834-4e7b-bce9-cfdff696509b None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:31.838757 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4c86b5-95a8-4236-8a28-8bee3341ce97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:31.839714 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4c86b5-95a8-4236-8a28-8bee3341ce97 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:31.840188 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4c86b5-95a8-4236-8a28-8bee3341ce97 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:31.840465 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4c86b5-95a8-4236-8a28-8bee3341ce97 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:31.847624 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4c86b5-95a8-4236-8a28-8bee3341ce97 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:31.849080 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4c86b5-95a8-4236-8a28-8bee3341ce97 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:31.862938 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application [None req-1f4c86b5-95a8-4236-8a28-8bee3341ce97 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 10:35:31.862938 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:35:31.862938 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:35:31.862938 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:35:31.862938 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:35:31.862938 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:35:31.862938 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:35:31.862938 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:35:31.862938 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:35:31.862938 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:35:31.862938 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:35:31.862938 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:35:31.862938 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 10:35:31.862938 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 10:35:31.862938 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 10:35:31.863870 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 10:35:31.863870 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:35:31.863870 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:35:31.863870 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:35:31.863870 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:35:31.863870 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:35:31.863870 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:35:31.863870 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:35:31.863870 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application with Lock( Jul 27 10:35:31.863870 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:35:31.863870 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:35:31.863870 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:35:31.863870 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:35:31.863870 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:35:31.863870 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:35:31.863870 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:35:31.863870 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:35:31.864731 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 10:35:31.864731 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 10:35:31.864731 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 10:35:31.864731 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 10:35:31.864731 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 10:35:31.864731 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 10:35:31.864731 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 10:35:31.864731 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Jul 27 10:35:31.864731 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 441/869] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Jul 27 10:35:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 10:35:31.866224 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4c86b5-95a8-4236-8a28-8bee3341ce97 None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:31.919032 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ad6e0e-d834-4e7b-bce9-cfdff696509b None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:31.959475 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba3b74c-1f9a-4296-83f1-10c70a917fe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:31.960139 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba3b74c-1f9a-4296-83f1-10c70a917fe0 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:31.960139 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba3b74c-1f9a-4296-83f1-10c70a917fe0 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:31.960746 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba3b74c-1f9a-4296-83f1-10c70a917fe0 None admin] PATH_INFO: `/v3/services` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:31.961365 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba3b74c-1f9a-4296-83f1-10c70a917fe0 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=86828) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:35:31.961675 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba3b74c-1f9a-4296-83f1-10c70a917fe0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:31.963092 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba3b74c-1f9a-4296-83f1-10c70a917fe0 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:31.967616 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 429/870] 10.222.0.36 () {56 vars in 1126 bytes} [Wed Jul 27 10:35:31 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:35:31.968382 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba3b74c-1f9a-4296-83f1-10c70a917fe0 None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:31.997334 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4c86b5-95a8-4236-8a28-8bee3341ce97 None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:32.039770 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c425f5-0d3e-4236-a4dd-f3424c8e106d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:32.040162 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c425f5-0d3e-4236-a4dd-f3424c8e106d None admin] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:32.040441 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c425f5-0d3e-4236-a4dd-f3424c8e106d None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:32.040871 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c425f5-0d3e-4236-a4dd-f3424c8e106d None admin] PATH_INFO: `/v3/services` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:32.041670 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c425f5-0d3e-4236-a4dd-f3424c8e106d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:32.043344 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c425f5-0d3e-4236-a4dd-f3424c8e106d None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:32.059355 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:32.070638 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 442/871] 10.222.0.36 () {60 vars in 1141 bytes} [Wed Jul 27 10:35:31 2022] POST /identity/v3/services => generated 251 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 10:35:32.130378 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba3b74c-1f9a-4296-83f1-10c70a917fe0 None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:32.179787 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb75efcf-036b-44f1-9625-ab14fe59a612 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:32.180398 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb75efcf-036b-44f1-9625-ab14fe59a612 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:32.180681 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb75efcf-036b-44f1-9625-ab14fe59a612 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:32.180953 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb75efcf-036b-44f1-9625-ab14fe59a612 None admin] PATH_INFO: `/v3/projects` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:32.181864 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb75efcf-036b-44f1-9625-ab14fe59a612 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86828) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:35:32.182176 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb75efcf-036b-44f1-9625-ab14fe59a612 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:32.183671 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb75efcf-036b-44f1-9625-ab14fe59a612 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:32.198913 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 430/872] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Jul 27 10:35:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:35:32.207749 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c425f5-0d3e-4236-a4dd-f3424c8e106d None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:32.230706 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:32.333010 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c425f5-0d3e-4236-a4dd-f3424c8e106d None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:32.369572 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3092f1be-ae83-4639-a0fc-ed223edd9e8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:32.370200 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3092f1be-ae83-4639-a0fc-ed223edd9e8e None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:32.370486 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3092f1be-ae83-4639-a0fc-ed223edd9e8e None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:32.370802 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3092f1be-ae83-4639-a0fc-ed223edd9e8e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:32.371608 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3092f1be-ae83-4639-a0fc-ed223edd9e8e None admin] RBAC: Adding query filter params (role.id=21156560eb6b436bb567a40b4976f3f4, scope.project.id=5f296d8e3b8f4001909addc49d8dac01, user.id=89ac506eefcd4e42a84b44fa15fb3975) {{(pid=86827) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:35:32.371980 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3092f1be-ae83-4639-a0fc-ed223edd9e8e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:32.373424 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3092f1be-ae83-4639-a0fc-ed223edd9e8e None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:32.380254 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 443/873] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Jul 27 10:35:32 2022] GET /identity/v3/role_assignments?user.id=89ac506eefcd4e42a84b44fa15fb3975&scope.project.id=5f296d8e3b8f4001909addc49d8dac01&role.id=21156560eb6b436bb567a40b4976f3f4 => generated 608 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:35:33.215977 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ab5966-ec19-465e-ad73-6ff2792d4c80 None None] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:33.216460 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ab5966-ec19-465e-ad73-6ff2792d4c80 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:33.216460 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ab5966-ec19-465e-ad73-6ff2792d4c80 None None] PATH_INFO: `/` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:33.218041 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 431/874] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 10:35:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 10:35:33.223239 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a2fa15-f953-46fd-a13f-b99290d6ce0b None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:33.223734 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a2fa15-f953-46fd-a13f-b99290d6ce0b None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:33.223816 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a2fa15-f953-46fd-a13f-b99290d6ce0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:33.242939 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-90a2fa15-f953-46fd-a13f-b99290d6ce0b None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:35:33.244157 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-90a2fa15-f953-46fd-a13f-b99290d6ce0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:35:33.318566 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 444/875] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 10:35:33 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 10:35:33.341196 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ab5966-ec19-465e-ad73-6ff2792d4c80 None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:33.446953 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ab5966-ec19-465e-ad73-6ff2792d4c80 None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:33.456078 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:33.485124 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1622f3c8-5d1b-482e-90aa-a58e382a6721 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:33.485755 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1622f3c8-5d1b-482e-90aa-a58e382a6721 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:33.486037 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1622f3c8-5d1b-482e-90aa-a58e382a6721 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:33.486344 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1622f3c8-5d1b-482e-90aa-a58e382a6721 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:33.497291 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1622f3c8-5d1b-482e-90aa-a58e382a6721 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 10:35:33.502302 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application [None req-1622f3c8-5d1b-482e-90aa-a58e382a6721 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 27 10:35:33.502302 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:35:33.502302 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:35:33.502302 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:35:33.502302 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:35:33.502302 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:35:33.502302 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:35:33.502302 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:35:33.502302 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:35:33.502302 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:35:33.502302 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:35:33.502302 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:35:33.502302 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 10:35:33.502302 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 10:35:33.502302 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 10:35:33.502302 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 10:35:33.503275 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 10:35:33.503275 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 10:35:33.503275 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 10:35:33.503275 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 10:35:33.503275 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:35:33.503275 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:35:33.503275 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:35:33.503275 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:35:33.503275 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:35:33.503275 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:35:33.503275 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:35:33.503275 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application with Lock( Jul 27 10:35:33.503275 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:35:33.503275 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:35:33.503275 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:35:33.503275 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:35:33.504116 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:35:33.504116 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:35:33.504116 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:35:33.504116 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:35:33.504116 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 10:35:33.504116 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 10:35:33.504116 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 10:35:33.504116 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 10:35:33.504116 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 10:35:33.504116 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 10:35:33.504116 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 27 10:35:33.504116 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Jul 27 10:35:33.504116 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 432/876] 10.222.0.36 () {56 vars in 1120 bytes} [Wed Jul 27 10:35:33 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 10:35:33.509956 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a2fa15-f953-46fd-a13f-b99290d6ce0b None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:33.525670 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:33.622062 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a2fa15-f953-46fd-a13f-b99290d6ce0b None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:33.658796 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd698db6-d43f-4fd1-b1d4-02d6b07ecaf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:33.659776 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd698db6-d43f-4fd1-b1d4-02d6b07ecaf0 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:33.660294 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd698db6-d43f-4fd1-b1d4-02d6b07ecaf0 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:33.660772 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd698db6-d43f-4fd1-b1d4-02d6b07ecaf0 None admin] PATH_INFO: `/v3/roles` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:33.661698 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd698db6-d43f-4fd1-b1d4-02d6b07ecaf0 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=86827) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:35:33.662245 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd698db6-d43f-4fd1-b1d4-02d6b07ecaf0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:33.663870 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd698db6-d43f-4fd1-b1d4-02d6b07ecaf0 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:33.673424 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 445/877] 10.222.0.36 () {56 vars in 1115 bytes} [Wed Jul 27 10:35:33 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 10:35:33.678939 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1622f3c8-5d1b-482e-90aa-a58e382a6721 None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:33.788215 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1622f3c8-5d1b-482e-90aa-a58e382a6721 None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:33.824292 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567e1d86-7eda-4e0c-af86-d043f85b5699 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:33.825035 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567e1d86-7eda-4e0c-af86-d043f85b5699 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:33.825380 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567e1d86-7eda-4e0c-af86-d043f85b5699 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:33.825699 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567e1d86-7eda-4e0c-af86-d043f85b5699 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:33.837495 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567e1d86-7eda-4e0c-af86-d043f85b5699 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:33.838988 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567e1d86-7eda-4e0c-af86-d043f85b5699 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:33.853469 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application [None req-567e1d86-7eda-4e0c-af86-d043f85b5699 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 10:35:33.853469 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:35:33.853469 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:35:33.853469 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:35:33.853469 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:35:33.853469 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:35:33.853469 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:35:33.853469 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:35:33.853469 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:35:33.853469 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:35:33.853469 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:35:33.853469 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:35:33.853469 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 10:35:33.853469 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 10:35:33.853469 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 10:35:33.854386 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 10:35:33.854386 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:35:33.854386 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:35:33.854386 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:35:33.854386 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:35:33.854386 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:35:33.854386 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:35:33.854386 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:35:33.854386 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application with Lock( Jul 27 10:35:33.854386 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:35:33.854386 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:35:33.854386 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:35:33.854386 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:35:33.854386 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:35:33.854386 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:35:33.854386 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:35:33.854386 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:35:33.855339 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 10:35:33.855339 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 10:35:33.855339 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 10:35:33.855339 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Jul 27 10:35:33.855339 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 433/878] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 10:35:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 10:35:33.860277 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd698db6-d43f-4fd1-b1d4-02d6b07ecaf0 None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:33.935155 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:33.972180 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd698db6-d43f-4fd1-b1d4-02d6b07ecaf0 None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:34.011663 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce0c39d-3b3d-4f80-b865-ee0357fa1320 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:34.012541 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce0c39d-3b3d-4f80-b865-ee0357fa1320 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:34.012829 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce0c39d-3b3d-4f80-b865-ee0357fa1320 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:34.013101 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce0c39d-3b3d-4f80-b865-ee0357fa1320 None admin] PATH_INFO: `/v3/domains` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:34.014009 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce0c39d-3b3d-4f80-b865-ee0357fa1320 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86827) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:35:34.014312 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce0c39d-3b3d-4f80-b865-ee0357fa1320 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:34.015784 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce0c39d-3b3d-4f80-b865-ee0357fa1320 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:34.029443 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 446/879] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 10:35:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:35:34.035490 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567e1d86-7eda-4e0c-af86-d043f85b5699 None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:34.067071 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:34.070760 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5096bad6-15d0-411d-a8cb-eaf3d0d1cdc9 None None] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:34.071209 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5096bad6-15d0-411d-a8cb-eaf3d0d1cdc9 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:34.071626 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5096bad6-15d0-411d-a8cb-eaf3d0d1cdc9 None None] PATH_INFO: `/` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:34.073826 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 447/880] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 10:35:34 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 10:35:34.079241 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8ac15e-0615-42fb-b935-e3c4d520a756 None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:34.079692 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8ac15e-0615-42fb-b935-e3c4d520a756 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:34.080240 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8ac15e-0615-42fb-b935-e3c4d520a756 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:34.098755 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-1b8ac15e-0615-42fb-b935-e3c4d520a756 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:35:34.100312 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-1b8ac15e-0615-42fb-b935-e3c4d520a756 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:35:34.161434 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567e1d86-7eda-4e0c-af86-d043f85b5699 None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:34.186792 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 448/881] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 10:35:34 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 27 10:35:34.202588 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccce8bd8-b10c-4311-9c53-d367185d63f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:34.203620 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccce8bd8-b10c-4311-9c53-d367185d63f8 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:34.204155 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccce8bd8-b10c-4311-9c53-d367185d63f8 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:34.204690 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccce8bd8-b10c-4311-9c53-d367185d63f8 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:34.211456 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8ac15e-0615-42fb-b935-e3c4d520a756 None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:34.216860 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccce8bd8-b10c-4311-9c53-d367185d63f8 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:34.218321 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccce8bd8-b10c-4311-9c53-d367185d63f8 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:34.230377 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application [None req-ccce8bd8-b10c-4311-9c53-d367185d63f8 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 27 10:35:34.230377 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:35:34.230377 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:35:34.230377 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:35:34.230377 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:35:34.230377 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:35:34.230377 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:35:34.230377 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:35:34.230377 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:35:34.230377 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:35:34.230377 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:35:34.230377 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:35:34.230377 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 10:35:34.230377 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 10:35:34.230377 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 10:35:34.230377 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 10:35:34.231382 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:35:34.231382 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:35:34.231382 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 10:35:34.231382 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 10:35:34.231382 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 10:35:34.231382 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 10:35:34.231382 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:35:34.231382 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:35:34.231382 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:35:34.231382 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:35:34.231382 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:35:34.231382 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application with Lock( Jul 27 10:35:34.231382 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:35:34.231382 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:35:34.231382 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:35:34.231382 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:35:34.231382 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:35:34.231382 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:35:34.232307 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:35:34.232307 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:35:34.232307 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 10:35:34.232307 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 10:35:34.232307 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 10:35:34.232307 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 10:35:34.232307 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 10:35:34.232307 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 10:35:34.232307 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 27 10:35:34.232307 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Jul 27 10:35:34.232307 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 434/882] 10.222.0.36 () {56 vars in 1117 bytes} [Wed Jul 27 10:35:34 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 10:35:34.237469 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccce8bd8-b10c-4311-9c53-d367185d63f8 None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:34.276032 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:34.317042 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8ac15e-0615-42fb-b935-e3c4d520a756 None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:34.326773 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:34.346738 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccce8bd8-b10c-4311-9c53-d367185d63f8 None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:34.357631 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db67a13c-59fc-4e72-b513-8c516c9239e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:34.358918 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db67a13c-59fc-4e72-b513-8c516c9239e0 None admin] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:34.359344 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db67a13c-59fc-4e72-b513-8c516c9239e0 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:34.359924 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db67a13c-59fc-4e72-b513-8c516c9239e0 None admin] PATH_INFO: `/v3/services` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:34.361533 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db67a13c-59fc-4e72-b513-8c516c9239e0 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:34.364522 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db67a13c-59fc-4e72-b513-8c516c9239e0 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:34.382899 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900a1703-bb3d-4fdc-aeb3-20f95f6ee6d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:34.383521 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900a1703-bb3d-4fdc-aeb3-20f95f6ee6d4 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:34.384156 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900a1703-bb3d-4fdc-aeb3-20f95f6ee6d4 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:34.384391 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900a1703-bb3d-4fdc-aeb3-20f95f6ee6d4 None admin] PATH_INFO: `/v3/users` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:34.385246 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900a1703-bb3d-4fdc-aeb3-20f95f6ee6d4 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=86828) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:35:34.385636 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900a1703-bb3d-4fdc-aeb3-20f95f6ee6d4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:34.388285 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900a1703-bb3d-4fdc-aeb3-20f95f6ee6d4 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:34.394672 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 449/883] 10.222.0.36 () {60 vars in 1141 bytes} [Wed Jul 27 10:35:34 2022] POST /identity/v3/services => generated 269 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 10:35:34.394672 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa89db3-2dad-40da-a10d-6f996fda5310 None None] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:34.394672 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa89db3-2dad-40da-a10d-6f996fda5310 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:34.394672 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa89db3-2dad-40da-a10d-6f996fda5310 None None] PATH_INFO: `/` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:34.394672 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 450/884] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 10:35:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 10:35:34.399909 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c697fea-822e-48af-949e-e9203bc38d62 None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:34.400281 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c697fea-822e-48af-949e-e9203bc38d62 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:34.400630 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c697fea-822e-48af-949e-e9203bc38d62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:34.416351 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 435/885] 10.222.0.36 () {56 vars in 1149 bytes} [Wed Jul 27 10:35:34 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:35:34.423279 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900a1703-bb3d-4fdc-aeb3-20f95f6ee6d4 None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:34.430644 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-3c697fea-822e-48af-949e-e9203bc38d62 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:35:34.431417 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-3c697fea-822e-48af-949e-e9203bc38d62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:35:34.528789 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 451/886] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 10:35:34 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 10:35:34.546827 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c697fea-822e-48af-949e-e9203bc38d62 None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:34.578637 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900a1703-bb3d-4fdc-aeb3-20f95f6ee6d4 None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:34.605249 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:34.617965 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0044c429-6e46-49e2-8e61-47b07f4ae2b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:34.618613 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0044c429-6e46-49e2-8e61-47b07f4ae2b9 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:34.618923 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0044c429-6e46-49e2-8e61-47b07f4ae2b9 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:34.619203 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0044c429-6e46-49e2-8e61-47b07f4ae2b9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:34.628152 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:34.661692 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c697fea-822e-48af-949e-e9203bc38d62 None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:34.697417 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf41e77-cb80-4052-a07e-6e141af5d244 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:34.698071 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf41e77-cb80-4052-a07e-6e141af5d244 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:34.698350 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf41e77-cb80-4052-a07e-6e141af5d244 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:34.698644 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf41e77-cb80-4052-a07e-6e141af5d244 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:34.699295 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf41e77-cb80-4052-a07e-6e141af5d244 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:34.700690 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf41e77-cb80-4052-a07e-6e141af5d244 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:34.701844 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0044c429-6e46-49e2-8e61-47b07f4ae2b9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:34.703524 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0044c429-6e46-49e2-8e61-47b07f4ae2b9 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:34.710615 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application [None req-bbf41e77-cb80-4052-a07e-6e141af5d244 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 27 10:35:34.710615 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:35:34.710615 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 27 10:35:34.710615 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 27 10:35:34.710615 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 27 10:35:34.710615 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 27 10:35:34.710615 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 27 10:35:34.710615 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 10:35:34.710615 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 27 10:35:34.710615 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Jul 27 10:35:34.710615 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 10:35:34.710615 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Jul 27 10:35:34.710615 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:35:34.710615 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:35:34.710615 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:35:34.710615 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:35:34.710615 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:35:34.711882 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:35:34.711882 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:35:34.711882 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:35:34.711882 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:35:34.711882 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:35:34.711882 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:35:34.711882 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 27 10:35:34.711882 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 27 10:35:34.711882 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 27 10:35:34.711882 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 27 10:35:34.711882 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:35:34.711882 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:35:34.711882 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:35:34.711882 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:35:34.711882 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:35:34.711882 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:35:34.712967 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:35:34.712967 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application with Lock( Jul 27 10:35:34.712967 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:35:34.712967 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:35:34.712967 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:35:34.712967 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:35:34.712967 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:35:34.712967 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:35:34.712967 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:35:34.712967 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:35:34.712967 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 27 10:35:34.712967 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 10:35:34.712967 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 27 10:35:34.712967 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Jul 27 10:35:34.712967 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 452/887] 10.222.0.36 () {56 vars in 1114 bytes} [Wed Jul 27 10:35:34 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 10:35:34.713772 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828c0e21-b0b1-4f97-9630-074a073adf38 None None] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:34.713772 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828c0e21-b0b1-4f97-9630-074a073adf38 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:34.713904 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828c0e21-b0b1-4f97-9630-074a073adf38 None None] PATH_INFO: `/` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:34.719944 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 453/888] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 10:35:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 10:35:34.719944 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828c0e21-b0b1-4f97-9630-074a073adf38 None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:34.819342 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 436/889] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jul 27 10:35:34 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 27 10:35:34.820597 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7244aed2-013f-4964-9283-a9fae5155216 None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:34.820941 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7244aed2-013f-4964-9283-a9fae5155216 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:34.821273 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7244aed2-013f-4964-9283-a9fae5155216 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:34.832894 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828c0e21-b0b1-4f97-9630-074a073adf38 None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:34.840346 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-7244aed2-013f-4964-9283-a9fae5155216 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:35:34.841536 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-7244aed2-013f-4964-9283-a9fae5155216 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:35:34.847362 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:34.871240 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f7d68a-85ae-4f23-adcc-98354fbe625c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:34.872383 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f7d68a-85ae-4f23-adcc-98354fbe625c None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:34.873071 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f7d68a-85ae-4f23-adcc-98354fbe625c None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:34.873883 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f7d68a-85ae-4f23-adcc-98354fbe625c None admin] PATH_INFO: `/v3/services` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:34.875074 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f7d68a-85ae-4f23-adcc-98354fbe625c None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=86827) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:35:34.875553 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f7d68a-85ae-4f23-adcc-98354fbe625c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:34.877063 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:34.878930 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f7d68a-85ae-4f23-adcc-98354fbe625c None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:34.885995 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 454/890] 10.222.0.36 () {56 vars in 1114 bytes} [Wed Jul 27 10:35:34 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 10:35:34.890656 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f7d68a-85ae-4f23-adcc-98354fbe625c None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:34.922887 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 437/891] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 10:35:34 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 10:35:34.924088 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7244aed2-013f-4964-9283-a9fae5155216 None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:35.001632 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f7d68a-85ae-4f23-adcc-98354fbe625c None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:35.038063 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3865c45-6a9b-4aba-8cd4-38a6269b1bc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:35.038841 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3865c45-6a9b-4aba-8cd4-38a6269b1bc7 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:35.039406 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3865c45-6a9b-4aba-8cd4-38a6269b1bc7 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:35.039406 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3865c45-6a9b-4aba-8cd4-38a6269b1bc7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:35.048253 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7244aed2-013f-4964-9283-a9fae5155216 None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:35.050442 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3865c45-6a9b-4aba-8cd4-38a6269b1bc7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:35.051994 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3865c45-6a9b-4aba-8cd4-38a6269b1bc7 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:35.066320 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application [None req-d3865c45-6a9b-4aba-8cd4-38a6269b1bc7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 10:35:35.066320 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:35:35.066320 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:35:35.066320 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:35:35.066320 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:35:35.066320 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:35:35.066320 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:35:35.066320 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:35:35.066320 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:35:35.066320 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:35:35.066320 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:35:35.066320 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:35:35.066320 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 10:35:35.066320 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 10:35:35.066320 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 10:35:35.067442 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 10:35:35.067442 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:35:35.067442 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:35:35.067442 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:35:35.067442 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:35:35.067442 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:35:35.067442 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:35:35.067442 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:35:35.067442 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application with Lock( Jul 27 10:35:35.067442 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:35:35.067442 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:35:35.067442 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:35:35.067442 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:35:35.067442 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:35:35.067442 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:35:35.067442 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:35:35.067442 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:35:35.068409 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 10:35:35.068409 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 10:35:35.068409 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 10:35:35.068409 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Jul 27 10:35:35.068409 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 455/892] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 10:35:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 10:35:35.068741 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3865c45-6a9b-4aba-8cd4-38a6269b1bc7 None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:35.091249 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef76bcd7-7e05-44b8-9c46-4af66b7619a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:35.092102 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef76bcd7-7e05-44b8-9c46-4af66b7619a4 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:35.092389 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef76bcd7-7e05-44b8-9c46-4af66b7619a4 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:35.092670 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef76bcd7-7e05-44b8-9c46-4af66b7619a4 None admin] PATH_INFO: `/v3/services` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:35.093355 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef76bcd7-7e05-44b8-9c46-4af66b7619a4 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=86828) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:35:35.093669 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef76bcd7-7e05-44b8-9c46-4af66b7619a4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:35.095548 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef76bcd7-7e05-44b8-9c46-4af66b7619a4 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:35.101448 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 438/893] 10.222.0.36 () {56 vars in 1114 bytes} [Wed Jul 27 10:35:34 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:35:35.101963 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:35.102220 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef76bcd7-7e05-44b8-9c46-4af66b7619a4 None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:35.135554 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:35.191697 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3865c45-6a9b-4aba-8cd4-38a6269b1bc7 None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:35.223468 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef76bcd7-7e05-44b8-9c46-4af66b7619a4 None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:35.231571 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693958a5-191e-4035-ab9c-72628fe21f4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:35.232214 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693958a5-191e-4035-ab9c-72628fe21f4e None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:35.232498 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693958a5-191e-4035-ab9c-72628fe21f4e None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:35.232775 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693958a5-191e-4035-ab9c-72628fe21f4e None admin] PATH_INFO: `/v3/services/object-store` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:35.233412 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693958a5-191e-4035-ab9c-72628fe21f4e None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:35.234810 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693958a5-191e-4035-ab9c-72628fe21f4e None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:35.244148 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application [None req-693958a5-191e-4035-ab9c-72628fe21f4e None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 27 10:35:35.244148 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:35:35.244148 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 27 10:35:35.244148 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 27 10:35:35.244148 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 27 10:35:35.244148 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 27 10:35:35.244148 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 27 10:35:35.244148 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 10:35:35.244148 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 27 10:35:35.244148 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Jul 27 10:35:35.244148 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 10:35:35.244148 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Jul 27 10:35:35.244148 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:35:35.244148 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:35:35.244148 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:35:35.244148 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:35:35.244148 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:35:35.245107 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:35:35.245107 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:35:35.245107 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:35:35.245107 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:35:35.245107 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:35:35.245107 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:35:35.245107 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 27 10:35:35.245107 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 27 10:35:35.245107 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 27 10:35:35.245107 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 27 10:35:35.245107 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:35:35.245107 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:35:35.245107 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:35:35.245107 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:35:35.245107 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:35:35.245107 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:35:35.245928 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:35:35.245928 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application with Lock( Jul 27 10:35:35.245928 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:35:35.245928 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:35:35.245928 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:35:35.245928 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:35:35.245928 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:35:35.245928 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:35:35.245928 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:35:35.245928 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:35:35.245928 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 27 10:35:35.245928 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 10:35:35.245928 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 27 10:35:35.245928 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Jul 27 10:35:35.245928 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 456/894] 10.222.0.36 () {56 vars in 1126 bytes} [Wed Jul 27 10:35:35 2022] GET /identity/v3/services/object-store => generated 93 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 10:35:35.247197 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693958a5-191e-4035-ab9c-72628fe21f4e None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:35.270780 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9206da3e-596a-4db2-b6a8-161d96567c7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:35.271402 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9206da3e-596a-4db2-b6a8-161d96567c7a None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:35.272255 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9206da3e-596a-4db2-b6a8-161d96567c7a None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:35.272514 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9206da3e-596a-4db2-b6a8-161d96567c7a None admin] PATH_INFO: `/v3/domains` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:35.273207 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9206da3e-596a-4db2-b6a8-161d96567c7a None admin] RBAC: Adding query filter params (name=Default) {{(pid=86828) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:35:35.273790 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9206da3e-596a-4db2-b6a8-161d96567c7a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:35.274938 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9206da3e-596a-4db2-b6a8-161d96567c7a None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:35.290702 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 439/895] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 10:35:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:35:35.290702 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9206da3e-596a-4db2-b6a8-161d96567c7a None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:35.343835 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:35.353142 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693958a5-191e-4035-ab9c-72628fe21f4e None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:35.387026 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:35.387723 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec949f9-d73b-46d5-8847-2c29b5078fda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:35.388349 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec949f9-d73b-46d5-8847-2c29b5078fda None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:35.388634 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec949f9-d73b-46d5-8847-2c29b5078fda None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:35.388997 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec949f9-d73b-46d5-8847-2c29b5078fda None admin] PATH_INFO: `/v3/endpoints` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:35.389726 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec949f9-d73b-46d5-8847-2c29b5078fda None admin] RBAC: Adding query filter params (interface=public, service_id=0f2ee153279647908ff233dc0475039f, region_id=RegionOne) {{(pid=86827) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:35:35.390092 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec949f9-d73b-46d5-8847-2c29b5078fda None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:35.391502 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec949f9-d73b-46d5-8847-2c29b5078fda None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:35.397429 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 457/896] 10.222.0.36 () {56 vars in 1251 bytes} [Wed Jul 27 10:35:35 2022] GET /identity/v3/endpoints?service_id=0f2ee153279647908ff233dc0475039f&interface=public®ion_id=RegionOne => generated 194 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:35:35.397989 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9206da3e-596a-4db2-b6a8-161d96567c7a None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:35.398203 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec949f9-d73b-46d5-8847-2c29b5078fda None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:35.438262 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a345544-c666-46ea-9f1a-0805344f4c0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:35.438963 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a345544-c666-46ea-9f1a-0805344f4c0a None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:35.439304 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a345544-c666-46ea-9f1a-0805344f4c0a None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:35.439583 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a345544-c666-46ea-9f1a-0805344f4c0a None admin] PATH_INFO: `/v3/services` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:35.440270 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a345544-c666-46ea-9f1a-0805344f4c0a None admin] RBAC: Adding query filter params (name=object-store) {{(pid=86828) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:35:35.440569 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a345544-c666-46ea-9f1a-0805344f4c0a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:35.441951 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a345544-c666-46ea-9f1a-0805344f4c0a None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:35.448288 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 440/897] 10.222.0.36 () {56 vars in 1122 bytes} [Wed Jul 27 10:35:35 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:35:35.448884 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a345544-c666-46ea-9f1a-0805344f4c0a None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:35.535022 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec949f9-d73b-46d5-8847-2c29b5078fda None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:35.561687 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a345544-c666-46ea-9f1a-0805344f4c0a None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:35.569941 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73ebcaf-5171-4a38-8b64-052cd7865ffa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:35.570587 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73ebcaf-5171-4a38-8b64-052cd7865ffa None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:35.570897 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73ebcaf-5171-4a38-8b64-052cd7865ffa None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:35.571172 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73ebcaf-5171-4a38-8b64-052cd7865ffa None admin] PATH_INFO: `/v3/projects/service` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:35.578081 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73ebcaf-5171-4a38-8b64-052cd7865ffa None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:35.579560 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73ebcaf-5171-4a38-8b64-052cd7865ffa None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:35.591099 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application [None req-a73ebcaf-5171-4a38-8b64-052cd7865ffa None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 10:35:35.591099 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:35:35.591099 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:35:35.591099 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:35:35.591099 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:35:35.591099 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:35:35.591099 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:35:35.591099 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:35:35.591099 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:35:35.591099 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:35:35.591099 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:35:35.591099 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:35:35.591099 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 10:35:35.591099 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 10:35:35.591099 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 10:35:35.592325 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 10:35:35.592325 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:35:35.592325 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:35:35.592325 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:35:35.592325 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:35:35.592325 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:35:35.592325 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:35:35.592325 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:35:35.592325 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application with Lock( Jul 27 10:35:35.592325 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:35:35.592325 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:35:35.592325 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:35:35.592325 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:35:35.592325 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:35:35.592325 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:35:35.592325 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:35:35.592325 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:35:35.593215 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 10:35:35.593215 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 10:35:35.593215 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 10:35:35.593215 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 10:35:35.593215 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 10:35:35.593215 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 10:35:35.593215 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 10:35:35.593215 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Jul 27 10:35:35.593215 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 458/898] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Jul 27 10:35:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 10:35:35.593215 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73ebcaf-5171-4a38-8b64-052cd7865ffa None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:35.605728 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3344397c-9984-49de-8068-69c5d9639628 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:35.606366 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3344397c-9984-49de-8068-69c5d9639628 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:35.606788 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3344397c-9984-49de-8068-69c5d9639628 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:35.607123 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3344397c-9984-49de-8068-69c5d9639628 None admin] PATH_INFO: `/v3/services` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:35.607847 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3344397c-9984-49de-8068-69c5d9639628 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:35.609273 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3344397c-9984-49de-8068-69c5d9639628 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:35.613390 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:35.626641 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 441/899] 10.222.0.36 () {56 vars in 1087 bytes} [Wed Jul 27 10:35:35 2022] GET /identity/v3/services => generated 1268 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 10:35:35.626641 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3344397c-9984-49de-8068-69c5d9639628 None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:35.657177 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:35.716742 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73ebcaf-5171-4a38-8b64-052cd7865ffa None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:35.753849 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3344397c-9984-49de-8068-69c5d9639628 None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:35.769025 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c938aa2-5a8c-462e-b6a7-9b85a874117c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:35.770455 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c938aa2-5a8c-462e-b6a7-9b85a874117c None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:35.770939 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c938aa2-5a8c-462e-b6a7-9b85a874117c None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:35.772518 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c938aa2-5a8c-462e-b6a7-9b85a874117c None admin] PATH_INFO: `/v3/services` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:35.773657 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c938aa2-5a8c-462e-b6a7-9b85a874117c None admin] RBAC: Adding query filter params (type=object-store) {{(pid=86827) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:35:35.774132 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c938aa2-5a8c-462e-b6a7-9b85a874117c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:35.776093 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c938aa2-5a8c-462e-b6a7-9b85a874117c None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:35.782703 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 459/900] 10.222.0.36 () {56 vars in 1122 bytes} [Wed Jul 27 10:35:35 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:35:35.796070 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a87d6f9-59a6-4dd3-8ab7-4e46c3f87a95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:35.796917 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a87d6f9-59a6-4dd3-8ab7-4e46c3f87a95 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:35.797305 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a87d6f9-59a6-4dd3-8ab7-4e46c3f87a95 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:35.797684 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a87d6f9-59a6-4dd3-8ab7-4e46c3f87a95 None admin] PATH_INFO: `/v3/projects` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:35.798583 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a87d6f9-59a6-4dd3-8ab7-4e46c3f87a95 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86828) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:35:35.799045 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a87d6f9-59a6-4dd3-8ab7-4e46c3f87a95 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:35.800700 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a87d6f9-59a6-4dd3-8ab7-4e46c3f87a95 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:35.817798 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 442/901] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Jul 27 10:35:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:35:35.823916 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c938aa2-5a8c-462e-b6a7-9b85a874117c None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:35.926094 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:35.951541 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c938aa2-5a8c-462e-b6a7-9b85a874117c None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:35.988634 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300a59c5-028e-4094-bfeb-b0d999a97fc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:35.989267 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300a59c5-028e-4094-bfeb-b0d999a97fc6 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:35.989550 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300a59c5-028e-4094-bfeb-b0d999a97fc6 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:35.989828 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300a59c5-028e-4094-bfeb-b0d999a97fc6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:35.990692 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300a59c5-028e-4094-bfeb-b0d999a97fc6 None admin] RBAC: Adding query filter params (role.id=722097dd0463434b9c4ecaebe7ecf9ea, scope.project.id=5f296d8e3b8f4001909addc49d8dac01, user.id=3d9fed24b4ba44b0b5961a603c4362c4) {{(pid=86827) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:35:35.991010 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300a59c5-028e-4094-bfeb-b0d999a97fc6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:35.992459 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300a59c5-028e-4094-bfeb-b0d999a97fc6 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:35.998865 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 460/902] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Jul 27 10:35:35 2022] GET /identity/v3/role_assignments?user.id=3d9fed24b4ba44b0b5961a603c4362c4&scope.project.id=5f296d8e3b8f4001909addc49d8dac01&role.id=722097dd0463434b9c4ecaebe7ecf9ea => generated 259 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:35:36.451815 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c3cdd0-ee32-44c1-9f5c-1185e7898527 None None] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:36.452352 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c3cdd0-ee32-44c1-9f5c-1185e7898527 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:36.452352 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c3cdd0-ee32-44c1-9f5c-1185e7898527 None None] PATH_INFO: `/` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:36.453980 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 443/903] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 10:35:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 10:35:36.459510 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f3c514-d863-4f59-887e-f8bc3cf90c61 None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:36.459937 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f3c514-d863-4f59-887e-f8bc3cf90c61 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:36.460118 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f3c514-d863-4f59-887e-f8bc3cf90c61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:36.478468 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-a5f3c514-d863-4f59-887e-f8bc3cf90c61 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:35:36.479695 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-a5f3c514-d863-4f59-887e-f8bc3cf90c61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:35:36.553493 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 461/904] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 10:35:36 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 10:35:36.575917 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c3cdd0-ee32-44c1-9f5c-1185e7898527 None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:36.664675 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:36.685873 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c3cdd0-ee32-44c1-9f5c-1185e7898527 None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:36.722210 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe9cfc0-17da-4d53-b6bc-f098c65d7f66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:36.722814 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe9cfc0-17da-4d53-b6bc-f098c65d7f66 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:36.723106 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe9cfc0-17da-4d53-b6bc-f098c65d7f66 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:36.723383 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe9cfc0-17da-4d53-b6bc-f098c65d7f66 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:36.724014 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe9cfc0-17da-4d53-b6bc-f098c65d7f66 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:36.725444 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe9cfc0-17da-4d53-b6bc-f098c65d7f66 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:36.734926 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application [None req-7fe9cfc0-17da-4d53-b6bc-f098c65d7f66 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 27 10:35:36.734926 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:35:36.734926 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 27 10:35:36.734926 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 27 10:35:36.734926 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 27 10:35:36.734926 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 27 10:35:36.734926 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 27 10:35:36.734926 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 10:35:36.734926 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 27 10:35:36.734926 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Jul 27 10:35:36.734926 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 10:35:36.734926 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Jul 27 10:35:36.734926 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:35:36.734926 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:35:36.734926 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:35:36.734926 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:35:36.735977 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:35:36.735977 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:35:36.735977 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:35:36.735977 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:35:36.735977 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:35:36.735977 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:35:36.735977 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:35:36.735977 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 27 10:35:36.735977 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 27 10:35:36.735977 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 27 10:35:36.735977 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 27 10:35:36.735977 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:35:36.735977 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:35:36.735977 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:35:36.735977 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:35:36.735977 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:35:36.736817 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:35:36.736817 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:35:36.736817 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application with Lock( Jul 27 10:35:36.736817 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:35:36.736817 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:35:36.736817 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:35:36.736817 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:35:36.736817 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:35:36.736817 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:35:36.736817 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:35:36.736817 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:35:36.736817 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 27 10:35:36.736817 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 10:35:36.736817 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 27 10:35:36.736817 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Jul 27 10:35:36.736817 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 444/905] 10.222.0.36 () {56 vars in 1132 bytes} [Wed Jul 27 10:35:36 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 10:35:36.742250 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f3c514-d863-4f59-887e-f8bc3cf90c61 None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:36.820614 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:36.849526 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f3c514-d863-4f59-887e-f8bc3cf90c61 None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:36.886036 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0e9cfb-280a-46ba-a3ab-be1d92d597e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:36.886700 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0e9cfb-280a-46ba-a3ab-be1d92d597e3 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:36.887046 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0e9cfb-280a-46ba-a3ab-be1d92d597e3 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:36.887326 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0e9cfb-280a-46ba-a3ab-be1d92d597e3 None admin] PATH_INFO: `/v3/services` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:36.888066 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0e9cfb-280a-46ba-a3ab-be1d92d597e3 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=86827) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:35:36.888377 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0e9cfb-280a-46ba-a3ab-be1d92d597e3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:36.889758 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0e9cfb-280a-46ba-a3ab-be1d92d597e3 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:36.894890 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 462/906] 10.222.0.36 () {56 vars in 1126 bytes} [Wed Jul 27 10:35:36 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:35:36.900870 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe9cfc0-17da-4d53-b6bc-f098c65d7f66 None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:37.014780 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe9cfc0-17da-4d53-b6bc-f098c65d7f66 None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:37.049352 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d888b126-d927-44e9-8594-9cf3face79b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:37.050315 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d888b126-d927-44e9-8594-9cf3face79b3 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:37.050833 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d888b126-d927-44e9-8594-9cf3face79b3 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:37.051313 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d888b126-d927-44e9-8594-9cf3face79b3 None admin] PATH_INFO: `/v3/services` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:37.052188 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d888b126-d927-44e9-8594-9cf3face79b3 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=86828) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:35:37.053219 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d888b126-d927-44e9-8594-9cf3face79b3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:37.054921 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d888b126-d927-44e9-8594-9cf3face79b3 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:37.060334 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 445/907] 10.222.0.36 () {56 vars in 1126 bytes} [Wed Jul 27 10:35:36 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:35:37.066505 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0e9cfb-280a-46ba-a3ab-be1d92d597e3 None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:37.173684 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0e9cfb-280a-46ba-a3ab-be1d92d597e3 None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:37.208343 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0759abc9-d01a-46c9-8cee-47d2bf86af5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:37.209479 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0759abc9-d01a-46c9-8cee-47d2bf86af5f None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:37.209766 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0759abc9-d01a-46c9-8cee-47d2bf86af5f None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:37.210039 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0759abc9-d01a-46c9-8cee-47d2bf86af5f None admin] PATH_INFO: `/v3/endpoints` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:37.210817 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0759abc9-d01a-46c9-8cee-47d2bf86af5f None admin] RBAC: Adding query filter params (interface=public, service_id=5b43c3b75b71418cabcf027c554c1660, region_id=RegionOne) {{(pid=86827) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:35:37.211127 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0759abc9-d01a-46c9-8cee-47d2bf86af5f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:37.212482 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0759abc9-d01a-46c9-8cee-47d2bf86af5f None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:37.217996 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 463/908] 10.222.0.36 () {56 vars in 1251 bytes} [Wed Jul 27 10:35:37 2022] GET /identity/v3/endpoints?service_id=5b43c3b75b71418cabcf027c554c1660&interface=public®ion_id=RegionOne => generated 194 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:35:37.223530 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d888b126-d927-44e9-8594-9cf3face79b3 None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:37.235799 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:37.325910 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d888b126-d927-44e9-8594-9cf3face79b3 None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:37.360577 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ce4a0b-7ba6-40f2-b7ec-5fb379fc3d9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:37.361262 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ce4a0b-7ba6-40f2-b7ec-5fb379fc3d9c None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:37.361545 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ce4a0b-7ba6-40f2-b7ec-5fb379fc3d9c None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:37.361876 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ce4a0b-7ba6-40f2-b7ec-5fb379fc3d9c None admin] PATH_INFO: `/v3/services` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:37.362539 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ce4a0b-7ba6-40f2-b7ec-5fb379fc3d9c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:37.363942 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ce4a0b-7ba6-40f2-b7ec-5fb379fc3d9c None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:37.369988 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 446/909] 10.222.0.36 () {56 vars in 1087 bytes} [Wed Jul 27 10:35:37 2022] GET /identity/v3/services => generated 1268 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 10:35:37.785117 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d83015-36fe-477f-8930-3d23db4f84aa None None] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:37.785688 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d83015-36fe-477f-8930-3d23db4f84aa None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:37.785958 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d83015-36fe-477f-8930-3d23db4f84aa None None] PATH_INFO: `/` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:37.789471 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 464/910] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 10:35:37 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 10:35:37.797561 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e2d7c1-e4bc-462e-87ac-3b1a9b827162 None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:37.798092 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e2d7c1-e4bc-462e-87ac-3b1a9b827162 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:37.798092 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e2d7c1-e4bc-462e-87ac-3b1a9b827162 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:37.817906 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-02e2d7c1-e4bc-462e-87ac-3b1a9b827162 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:35:37.819613 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-02e2d7c1-e4bc-462e-87ac-3b1a9b827162 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:35:37.895072 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 447/911] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 10:35:37 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 10:35:37.910631 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cc6bd9-8b8b-4640-978b-a0c23ebf12c5 None None] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:37.911242 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cc6bd9-8b8b-4640-978b-a0c23ebf12c5 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:37.911711 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cc6bd9-8b8b-4640-978b-a0c23ebf12c5 None None] PATH_INFO: `/` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:37.913809 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 465/912] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 10:35:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 10:35:37.919215 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b902787-77a7-48cc-84cd-77cb1f473226 None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:37.919743 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b902787-77a7-48cc-84cd-77cb1f473226 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:37.920278 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b902787-77a7-48cc-84cd-77cb1f473226 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:37.922051 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cc6bd9-8b8b-4640-978b-a0c23ebf12c5 None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:37.925106 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:37.928318 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:37.944485 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-4b902787-77a7-48cc-84cd-77cb1f473226 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:35:37.946142 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-4b902787-77a7-48cc-84cd-77cb1f473226 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:35:38.032069 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cc6bd9-8b8b-4640-978b-a0c23ebf12c5 None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:38.038692 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 448/913] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 10:35:37 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 10:35:38.060061 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b902787-77a7-48cc-84cd-77cb1f473226 None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:38.081432 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4ad3db-c585-490f-b369-fc47ee43136b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:38.082230 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4ad3db-c585-490f-b369-fc47ee43136b None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:38.082593 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4ad3db-c585-490f-b369-fc47ee43136b None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:38.082997 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4ad3db-c585-490f-b369-fc47ee43136b None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:38.083757 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4ad3db-c585-490f-b369-fc47ee43136b None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:38.086414 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4ad3db-c585-490f-b369-fc47ee43136b None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:38.096926 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application [None req-7a4ad3db-c585-490f-b369-fc47ee43136b None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 27 10:35:38.096926 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:35:38.096926 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 27 10:35:38.096926 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 27 10:35:38.096926 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 27 10:35:38.096926 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 27 10:35:38.096926 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 27 10:35:38.096926 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 10:35:38.096926 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 27 10:35:38.096926 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Jul 27 10:35:38.096926 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 10:35:38.096926 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Jul 27 10:35:38.096926 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:35:38.096926 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:35:38.096926 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:35:38.096926 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:35:38.096926 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:35:38.097928 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:35:38.097928 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:35:38.097928 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:35:38.097928 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:35:38.097928 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:35:38.097928 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:35:38.097928 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 27 10:35:38.097928 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 27 10:35:38.097928 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 27 10:35:38.097928 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 27 10:35:38.097928 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:35:38.097928 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:35:38.097928 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:35:38.097928 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:35:38.097928 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:35:38.097928 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:35:38.098789 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:35:38.098789 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application with Lock( Jul 27 10:35:38.098789 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:35:38.098789 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:35:38.098789 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:35:38.098789 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:35:38.098789 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:35:38.098789 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:35:38.098789 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:35:38.098789 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:35:38.098789 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 27 10:35:38.098789 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 10:35:38.098789 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 27 10:35:38.098789 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Jul 27 10:35:38.103323 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 466/914] 10.222.0.36 () {56 vars in 1114 bytes} [Wed Jul 27 10:35:37 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 10:35:38.104833 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4ad3db-c585-490f-b369-fc47ee43136b None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:38.181969 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:38.200936 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b902787-77a7-48cc-84cd-77cb1f473226 None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:38.223980 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4ad3db-c585-490f-b369-fc47ee43136b None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:38.248687 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:38.254487 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4d3932-df02-4a56-b32c-a662e7f02809 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:38.255149 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4d3932-df02-4a56-b32c-a662e7f02809 None admin] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:38.255433 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4d3932-df02-4a56-b32c-a662e7f02809 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:38.255709 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4d3932-df02-4a56-b32c-a662e7f02809 None admin] PATH_INFO: `/v3/services` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:38.256948 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4d3932-df02-4a56-b32c-a662e7f02809 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:38.258429 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4d3932-df02-4a56-b32c-a662e7f02809 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:38.266663 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81509994-2672-4992-95a1-3a8b1aa0acba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:38.267642 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81509994-2672-4992-95a1-3a8b1aa0acba None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:38.267944 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81509994-2672-4992-95a1-3a8b1aa0acba None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:38.268385 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81509994-2672-4992-95a1-3a8b1aa0acba None admin] PATH_INFO: `/v3/services` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:38.269031 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81509994-2672-4992-95a1-3a8b1aa0acba None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=86827) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:35:38.269337 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81509994-2672-4992-95a1-3a8b1aa0acba None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:38.270708 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81509994-2672-4992-95a1-3a8b1aa0acba None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:38.280816 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 449/915] 10.222.0.36 () {60 vars in 1141 bytes} [Wed Jul 27 10:35:38 2022] POST /identity/v3/services => generated 241 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 10:35:38.280816 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5650670c-9c7c-4341-837c-32705419f1d2 None None] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:38.280816 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5650670c-9c7c-4341-837c-32705419f1d2 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:38.280816 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5650670c-9c7c-4341-837c-32705419f1d2 None None] PATH_INFO: `/` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:38.280816 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 450/916] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 10:35:38 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 10:35:38.282559 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43b4122-6823-40cd-b372-98921637e9e6 None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:38.282880 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43b4122-6823-40cd-b372-98921637e9e6 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:38.283159 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43b4122-6823-40cd-b372-98921637e9e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:38.288653 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 467/917] 10.222.0.36 () {56 vars in 1114 bytes} [Wed Jul 27 10:35:38 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 10:35:38.300076 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81509994-2672-4992-95a1-3a8b1aa0acba None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:38.303417 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-c43b4122-6823-40cd-b372-98921637e9e6 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:35:38.304658 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-c43b4122-6823-40cd-b372-98921637e9e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:35:38.399084 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 451/918] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 10:35:38 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 27 10:35:38.416023 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43b4122-6823-40cd-b372-98921637e9e6 None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:38.448226 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81509994-2672-4992-95a1-3a8b1aa0acba None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:38.485323 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:38.489495 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ac9905-001b-4ace-8be9-f9781feb1762 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:38.491091 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ac9905-001b-4ace-8be9-f9781feb1762 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:38.494809 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ac9905-001b-4ace-8be9-f9781feb1762 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:38.494809 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ac9905-001b-4ace-8be9-f9781feb1762 None admin] PATH_INFO: `/v3/services` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:38.494809 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ac9905-001b-4ace-8be9-f9781feb1762 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=86827) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:35:38.494809 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ac9905-001b-4ace-8be9-f9781feb1762 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:38.494809 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ac9905-001b-4ace-8be9-f9781feb1762 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:38.504486 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 468/919] 10.222.0.36 () {56 vars in 1114 bytes} [Wed Jul 27 10:35:38 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:35:38.505717 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ac9905-001b-4ace-8be9-f9781feb1762 None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:38.547715 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43b4122-6823-40cd-b372-98921637e9e6 None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:38.573709 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:38.604349 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d552de7-5bda-4937-85a9-7b559ab059fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:38.605289 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d552de7-5bda-4937-85a9-7b559ab059fe None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:38.605729 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d552de7-5bda-4937-85a9-7b559ab059fe None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:38.606724 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d552de7-5bda-4937-85a9-7b559ab059fe None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:38.619700 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d552de7-5bda-4937-85a9-7b559ab059fe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 10:35:38.625322 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application [None req-6d552de7-5bda-4937-85a9-7b559ab059fe None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 27 10:35:38.625322 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:35:38.625322 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:35:38.625322 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:35:38.625322 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:35:38.625322 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:35:38.625322 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:35:38.625322 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:35:38.625322 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:35:38.625322 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:35:38.625322 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:35:38.625322 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:35:38.625322 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 10:35:38.625322 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 10:35:38.625322 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 10:35:38.625322 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 10:35:38.626447 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 10:35:38.626447 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 10:35:38.626447 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 10:35:38.626447 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 10:35:38.626447 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:35:38.626447 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:35:38.626447 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:35:38.626447 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:35:38.626447 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:35:38.626447 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:35:38.626447 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:35:38.626447 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application with Lock( Jul 27 10:35:38.626447 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:35:38.626447 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:35:38.626447 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:35:38.626447 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:35:38.627460 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:35:38.627460 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:35:38.627460 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:35:38.627460 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:35:38.627460 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 10:35:38.627460 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 10:35:38.627460 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 10:35:38.627460 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 10:35:38.627460 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 10:35:38.627460 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 10:35:38.627460 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 27 10:35:38.627460 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Jul 27 10:35:38.629799 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 452/920] 10.222.0.36 () {56 vars in 1120 bytes} [Wed Jul 27 10:35:38 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 10:35:38.636789 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d552de7-5bda-4937-85a9-7b559ab059fe None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:38.639063 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ac9905-001b-4ace-8be9-f9781feb1762 None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:38.687298 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1bc1b6-3913-4506-9e29-987589b7dd72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:38.687928 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1bc1b6-3913-4506-9e29-987589b7dd72 None admin] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:38.688686 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1bc1b6-3913-4506-9e29-987589b7dd72 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:38.688976 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1bc1b6-3913-4506-9e29-987589b7dd72 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:38.689844 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1bc1b6-3913-4506-9e29-987589b7dd72 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:38.691642 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1bc1b6-3913-4506-9e29-987589b7dd72 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:38.723184 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 469/921] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Jul 27 10:35:38 2022] POST /identity/v3/endpoints => generated 346 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 10:35:38.762769 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d552de7-5bda-4937-85a9-7b559ab059fe None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:38.809243 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3832e43a-940b-41de-bafb-670309741ded None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:38.810661 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3832e43a-940b-41de-bafb-670309741ded None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:38.811123 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3832e43a-940b-41de-bafb-670309741ded None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:38.811548 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3832e43a-940b-41de-bafb-670309741ded None admin] PATH_INFO: `/v3/roles` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:38.812440 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3832e43a-940b-41de-bafb-670309741ded None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=86828) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:35:38.812902 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3832e43a-940b-41de-bafb-670309741ded None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:38.814619 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3832e43a-940b-41de-bafb-670309741ded None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:38.824762 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 453/922] 10.222.0.36 () {56 vars in 1115 bytes} [Wed Jul 27 10:35:38 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:35:38.830947 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1bc1b6-3913-4506-9e29-987589b7dd72 None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:38.888735 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:38.949440 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1bc1b6-3913-4506-9e29-987589b7dd72 None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:38.990831 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b2510b-40b8-4afe-b4e6-62924a1013ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:38.991495 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b2510b-40b8-4afe-b4e6-62924a1013ca None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:38.991783 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b2510b-40b8-4afe-b4e6-62924a1013ca None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:38.992065 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b2510b-40b8-4afe-b4e6-62924a1013ca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:39.003507 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b2510b-40b8-4afe-b4e6-62924a1013ca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:39.004970 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b2510b-40b8-4afe-b4e6-62924a1013ca None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:39.021060 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application [None req-84b2510b-40b8-4afe-b4e6-62924a1013ca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 10:35:39.021060 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:35:39.021060 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:35:39.021060 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:35:39.021060 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:35:39.021060 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:35:39.021060 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:35:39.021060 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:35:39.021060 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:35:39.021060 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:35:39.021060 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:35:39.021060 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:35:39.021060 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 10:35:39.021060 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 10:35:39.021060 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 10:35:39.022575 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 10:35:39.022575 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:35:39.022575 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:35:39.022575 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:35:39.022575 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:35:39.022575 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:35:39.022575 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:35:39.022575 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:35:39.022575 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application with Lock( Jul 27 10:35:39.022575 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:35:39.022575 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:35:39.022575 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:35:39.022575 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:35:39.022575 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:35:39.022575 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:35:39.022575 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:35:39.022575 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:35:39.023615 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 10:35:39.023615 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 10:35:39.023615 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 10:35:39.023615 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Jul 27 10:35:39.023615 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 470/923] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 10:35:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 10:35:39.030035 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3832e43a-940b-41de-bafb-670309741ded None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:39.069390 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:39.140594 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3832e43a-940b-41de-bafb-670309741ded None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:39.175491 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e9ee58-c710-48c1-acff-b14f667dcfb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:39.176113 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e9ee58-c710-48c1-acff-b14f667dcfb9 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:39.176397 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e9ee58-c710-48c1-acff-b14f667dcfb9 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:39.176674 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e9ee58-c710-48c1-acff-b14f667dcfb9 None admin] PATH_INFO: `/v3/domains` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:39.177435 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e9ee58-c710-48c1-acff-b14f667dcfb9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86828) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:35:39.178004 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e9ee58-c710-48c1-acff-b14f667dcfb9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:39.179875 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e9ee58-c710-48c1-acff-b14f667dcfb9 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:39.192734 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 454/924] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 10:35:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:35:39.198862 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b2510b-40b8-4afe-b4e6-62924a1013ca None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:39.311586 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b2510b-40b8-4afe-b4e6-62924a1013ca None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:39.313128 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:39.349100 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf870aa-9f38-4fd1-b184-80c4031e51c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:39.349779 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf870aa-9f38-4fd1-b184-80c4031e51c1 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:39.350063 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf870aa-9f38-4fd1-b184-80c4031e51c1 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:39.350339 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf870aa-9f38-4fd1-b184-80c4031e51c1 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:39.358293 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf870aa-9f38-4fd1-b184-80c4031e51c1 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:39.359765 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf870aa-9f38-4fd1-b184-80c4031e51c1 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:39.371837 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application [None req-9cf870aa-9f38-4fd1-b184-80c4031e51c1 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 27 10:35:39.371837 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:35:39.371837 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:35:39.371837 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:35:39.371837 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:35:39.371837 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:35:39.371837 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:35:39.371837 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:35:39.371837 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:35:39.371837 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:35:39.371837 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:35:39.371837 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:35:39.371837 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 10:35:39.371837 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 10:35:39.371837 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 10:35:39.371837 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 10:35:39.372799 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:35:39.372799 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:35:39.372799 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 10:35:39.372799 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 10:35:39.372799 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 10:35:39.372799 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 10:35:39.372799 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:35:39.372799 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:35:39.372799 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:35:39.372799 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:35:39.372799 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:35:39.372799 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application with Lock( Jul 27 10:35:39.372799 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:35:39.372799 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:35:39.372799 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:35:39.372799 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:35:39.372799 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:35:39.372799 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:35:39.373734 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:35:39.373734 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:35:39.373734 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 10:35:39.373734 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 10:35:39.373734 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 10:35:39.373734 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 10:35:39.373734 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 10:35:39.373734 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 10:35:39.373734 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 27 10:35:39.373734 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Jul 27 10:35:39.373734 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 471/925] 10.222.0.36 () {56 vars in 1117 bytes} [Wed Jul 27 10:35:39 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 10:35:39.378982 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e9ee58-c710-48c1-acff-b14f667dcfb9 None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:39.456077 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b47b102-3a9f-40eb-ad12-e5be32c05cef None None] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:39.456377 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b47b102-3a9f-40eb-ad12-e5be32c05cef None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:39.456563 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b47b102-3a9f-40eb-ad12-e5be32c05cef None None] PATH_INFO: `/` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:39.458176 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 472/926] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 10:35:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 10:35:39.463251 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d05d52-b9da-4ef7-9c38-455965b6e38c None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:39.463537 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d05d52-b9da-4ef7-9c38-455965b6e38c None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:39.463827 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d05d52-b9da-4ef7-9c38-455965b6e38c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:39.479162 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:39.482635 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-f6d05d52-b9da-4ef7-9c38-455965b6e38c None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:35:39.483876 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-f6d05d52-b9da-4ef7-9c38-455965b6e38c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:35:39.487081 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e9ee58-c710-48c1-acff-b14f667dcfb9 None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:39.526817 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89977ed6-d183-4a64-b4d2-ea78abdda53f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:39.527742 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89977ed6-d183-4a64-b4d2-ea78abdda53f None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:39.528231 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89977ed6-d183-4a64-b4d2-ea78abdda53f None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:39.528653 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89977ed6-d183-4a64-b4d2-ea78abdda53f None admin] PATH_INFO: `/v3/users` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:39.530070 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89977ed6-d183-4a64-b4d2-ea78abdda53f None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=86828) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:35:39.530962 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89977ed6-d183-4a64-b4d2-ea78abdda53f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:39.532881 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89977ed6-d183-4a64-b4d2-ea78abdda53f None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:39.557496 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 455/927] 10.222.0.36 () {56 vars in 1149 bytes} [Wed Jul 27 10:35:39 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:35:39.563367 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 473/928] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 10:35:39 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 10:35:39.563646 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89977ed6-d183-4a64-b4d2-ea78abdda53f None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:39.585742 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d05d52-b9da-4ef7-9c38-455965b6e38c None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:39.673443 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89977ed6-d183-4a64-b4d2-ea78abdda53f None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:39.688434 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d05d52-b9da-4ef7-9c38-455965b6e38c None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:39.689846 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:39.709858 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0915de59-fc31-46f5-8923-3b1f0b37d909 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:39.710912 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0915de59-fc31-46f5-8923-3b1f0b37d909 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:39.711426 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0915de59-fc31-46f5-8923-3b1f0b37d909 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:39.711943 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0915de59-fc31-46f5-8923-3b1f0b37d909 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:39.722377 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0915de59-fc31-46f5-8923-3b1f0b37d909 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:39.724403 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0915de59-fc31-46f5-8923-3b1f0b37d909 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:39.726369 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb571d04-08b5-48c4-b265-f1f9e280c25c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:39.727058 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb571d04-08b5-48c4-b265-f1f9e280c25c None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:39.727436 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb571d04-08b5-48c4-b265-f1f9e280c25c None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:39.727687 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb571d04-08b5-48c4-b265-f1f9e280c25c None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:39.728317 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb571d04-08b5-48c4-b265-f1f9e280c25c None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:39.729773 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb571d04-08b5-48c4-b265-f1f9e280c25c None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:39.739063 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application [None req-eb571d04-08b5-48c4-b265-f1f9e280c25c None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 27 10:35:39.739063 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:35:39.739063 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 27 10:35:39.739063 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 27 10:35:39.739063 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 27 10:35:39.739063 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 27 10:35:39.739063 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 27 10:35:39.739063 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 10:35:39.739063 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 27 10:35:39.739063 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Jul 27 10:35:39.739063 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 10:35:39.739063 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Jul 27 10:35:39.739063 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:35:39.739063 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:35:39.739063 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:35:39.739063 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:35:39.739983 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:35:39.739983 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:35:39.739983 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:35:39.739983 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:35:39.739983 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:35:39.739983 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:35:39.739983 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:35:39.739983 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 27 10:35:39.739983 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 27 10:35:39.739983 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 27 10:35:39.739983 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 27 10:35:39.739983 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:35:39.739983 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:35:39.739983 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:35:39.739983 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:35:39.739983 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:35:39.740855 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:35:39.740855 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:35:39.740855 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application with Lock( Jul 27 10:35:39.740855 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:35:39.740855 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:35:39.740855 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:35:39.740855 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:35:39.740855 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:35:39.740855 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:35:39.740855 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:35:39.740855 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:35:39.740855 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 27 10:35:39.740855 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 10:35:39.740855 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 27 10:35:39.740855 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Jul 27 10:35:39.745396 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 474/929] 10.222.0.36 () {56 vars in 1132 bytes} [Wed Jul 27 10:35:39 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 10:35:39.745692 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application [None req-0915de59-fc31-46f5-8923-3b1f0b37d909 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 10:35:39.745692 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:35:39.745692 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:35:39.745692 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:35:39.745692 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:35:39.745692 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:35:39.745692 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:35:39.745692 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:35:39.745692 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:35:39.745692 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:35:39.745692 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:35:39.745692 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:35:39.745692 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 10:35:39.745692 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 10:35:39.745692 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 10:35:39.746558 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 10:35:39.746558 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:35:39.746558 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:35:39.746558 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:35:39.746558 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:35:39.746558 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:35:39.746558 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:35:39.746558 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:35:39.746558 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application with Lock( Jul 27 10:35:39.746558 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:35:39.746558 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:35:39.746558 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:35:39.746558 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:35:39.746558 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:35:39.746558 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:35:39.746558 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:35:39.746558 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:35:39.747523 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 10:35:39.747523 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 10:35:39.747523 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 10:35:39.747523 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Jul 27 10:35:39.747753 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb571d04-08b5-48c4-b265-f1f9e280c25c None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:39.750643 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 456/930] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 10:35:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 10:35:39.755293 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0915de59-fc31-46f5-8923-3b1f0b37d909 None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:39.765608 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:39.857456 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb571d04-08b5-48c4-b265-f1f9e280c25c None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:39.861403 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0915de59-fc31-46f5-8923-3b1f0b37d909 None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:39.897152 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ac53f0-a328-47c8-bab1-edb8fb23278b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:39.898061 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ac53f0-a328-47c8-bab1-edb8fb23278b None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:39.898768 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ac53f0-a328-47c8-bab1-edb8fb23278b None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:39.899460 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ac53f0-a328-47c8-bab1-edb8fb23278b None admin] PATH_INFO: `/v3/services` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:39.900304 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45436877-eacb-43c9-a5e6-7442b2c20e53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:39.901065 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45436877-eacb-43c9-a5e6-7442b2c20e53 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:39.901455 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45436877-eacb-43c9-a5e6-7442b2c20e53 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:39.901722 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ac53f0-a328-47c8-bab1-edb8fb23278b None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=86827) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:35:39.901921 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45436877-eacb-43c9-a5e6-7442b2c20e53 None admin] PATH_INFO: `/v3/domains` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:39.902710 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ac53f0-a328-47c8-bab1-edb8fb23278b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:39.902881 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45436877-eacb-43c9-a5e6-7442b2c20e53 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86828) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:35:39.903246 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45436877-eacb-43c9-a5e6-7442b2c20e53 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:39.904768 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45436877-eacb-43c9-a5e6-7442b2c20e53 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:39.906040 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ac53f0-a328-47c8-bab1-edb8fb23278b None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:39.918750 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 475/931] 10.222.0.36 () {56 vars in 1126 bytes} [Wed Jul 27 10:35:39 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 10:35:39.920393 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ac53f0-a328-47c8-bab1-edb8fb23278b None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:39.923555 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 457/932] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 10:35:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:35:39.924598 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45436877-eacb-43c9-a5e6-7442b2c20e53 None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:39.946738 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:40.006590 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:40.038780 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ac53f0-a328-47c8-bab1-edb8fb23278b None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:40.042868 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45436877-eacb-43c9-a5e6-7442b2c20e53 None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:40.081891 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae23916-9d75-435e-b591-6e3202e94cdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:40.082680 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017a5847-fcd4-4d5c-b11b-38f1e7ec7a93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:40.082877 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae23916-9d75-435e-b591-6e3202e94cdc None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:40.083137 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae23916-9d75-435e-b591-6e3202e94cdc None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:40.083313 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017a5847-fcd4-4d5c-b11b-38f1e7ec7a93 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:40.083404 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae23916-9d75-435e-b591-6e3202e94cdc None admin] PATH_INFO: `/v3/projects/service` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:40.083600 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017a5847-fcd4-4d5c-b11b-38f1e7ec7a93 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:40.083883 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017a5847-fcd4-4d5c-b11b-38f1e7ec7a93 None admin] PATH_INFO: `/v3/services` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:40.084621 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017a5847-fcd4-4d5c-b11b-38f1e7ec7a93 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=86827) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:35:40.084935 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017a5847-fcd4-4d5c-b11b-38f1e7ec7a93 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:40.086559 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017a5847-fcd4-4d5c-b11b-38f1e7ec7a93 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:40.090676 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae23916-9d75-435e-b591-6e3202e94cdc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:40.092194 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae23916-9d75-435e-b591-6e3202e94cdc None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:40.092413 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 476/933] 10.222.0.36 () {56 vars in 1126 bytes} [Wed Jul 27 10:35:39 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 10:35:40.099465 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017a5847-fcd4-4d5c-b11b-38f1e7ec7a93 None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:40.104051 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application [None req-cae23916-9d75-435e-b591-6e3202e94cdc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 10:35:40.104051 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:35:40.104051 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:35:40.104051 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:35:40.104051 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:35:40.104051 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:35:40.104051 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:35:40.104051 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:35:40.104051 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:35:40.104051 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:35:40.104051 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:35:40.104051 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:35:40.104051 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 10:35:40.104051 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 10:35:40.104051 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 10:35:40.105039 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 10:35:40.105039 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:35:40.105039 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:35:40.105039 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:35:40.105039 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:35:40.105039 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:35:40.105039 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:35:40.105039 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:35:40.105039 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application with Lock( Jul 27 10:35:40.105039 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:35:40.105039 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:35:40.105039 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:35:40.105039 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:35:40.105039 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:35:40.105039 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:35:40.105039 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:35:40.105039 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:35:40.106267 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 10:35:40.106267 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 10:35:40.106267 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 10:35:40.106267 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 10:35:40.106267 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 10:35:40.106267 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 10:35:40.106267 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 10:35:40.106267 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Jul 27 10:35:40.106267 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 458/934] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Jul 27 10:35:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 10:35:40.110872 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae23916-9d75-435e-b591-6e3202e94cdc None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:40.202973 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:40.211266 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017a5847-fcd4-4d5c-b11b-38f1e7ec7a93 None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:40.215241 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae23916-9d75-435e-b591-6e3202e94cdc None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:40.241819 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:40.251054 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f8cd31-fbee-44a8-9bc9-e5a8e5112054 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:40.251513 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f0344c-cd8b-4334-a5fa-8c59f1ab76da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:40.252150 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f0344c-cd8b-4334-a5fa-8c59f1ab76da None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:40.252429 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f0344c-cd8b-4334-a5fa-8c59f1ab76da None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:40.252656 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f8cd31-fbee-44a8-9bc9-e5a8e5112054 None admin] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:40.252769 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f0344c-cd8b-4334-a5fa-8c59f1ab76da None admin] PATH_INFO: `/v3/projects` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:40.253247 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f8cd31-fbee-44a8-9bc9-e5a8e5112054 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:40.253461 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f0344c-cd8b-4334-a5fa-8c59f1ab76da None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86828) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:35:40.253755 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f0344c-cd8b-4334-a5fa-8c59f1ab76da None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:40.254128 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f8cd31-fbee-44a8-9bc9-e5a8e5112054 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:40.255086 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f8cd31-fbee-44a8-9bc9-e5a8e5112054 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:40.255208 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f0344c-cd8b-4334-a5fa-8c59f1ab76da None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:40.256950 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f8cd31-fbee-44a8-9bc9-e5a8e5112054 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:40.276301 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 459/935] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Jul 27 10:35:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:35:40.277229 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f0344c-cd8b-4334-a5fa-8c59f1ab76da None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:40.281587 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 477/936] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Jul 27 10:35:40 2022] POST /identity/v3/endpoints => generated 347 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 10:35:40.367700 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12e2f91-af95-4788-8c76-004637dcb68a None None] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:40.368030 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12e2f91-af95-4788-8c76-004637dcb68a None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:40.368264 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12e2f91-af95-4788-8c76-004637dcb68a None None] PATH_INFO: `/` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:40.369895 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 478/937] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 10:35:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 10:35:40.375563 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193bde4a-a479-40d4-baa8-90daeaaff88b None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:40.375866 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193bde4a-a479-40d4-baa8-90daeaaff88b None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:40.376147 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193bde4a-a479-40d4-baa8-90daeaaff88b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:40.392395 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f0344c-cd8b-4334-a5fa-8c59f1ab76da None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:40.397334 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-193bde4a-a479-40d4-baa8-90daeaaff88b None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:35:40.398725 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-193bde4a-a479-40d4-baa8-90daeaaff88b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:35:40.437270 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea2aed7-5117-49f7-ba5e-ed66fe20b4ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:40.437953 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea2aed7-5117-49f7-ba5e-ed66fe20b4ac None admin] REQUEST_METHOD: `PUT` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:40.438236 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea2aed7-5117-49f7-ba5e-ed66fe20b4ac None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:40.438512 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea2aed7-5117-49f7-ba5e-ed66fe20b4ac None admin] PATH_INFO: `/v3/projects/5f296d8e3b8f4001909addc49d8dac01/users/3d9fed24b4ba44b0b5961a603c4362c4/roles/722097dd0463434b9c4ecaebe7ecf9ea` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:40.441357 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea2aed7-5117-49f7-ba5e-ed66fe20b4ac None admin] RBAC: Authorizing `identity:create_grant(project_id=5f296d8e3b8f4001909addc49d8dac01, user_id=3d9fed24b4ba44b0b5961a603c4362c4, role_id=722097dd0463434b9c4ecaebe7ecf9ea)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:40.442297 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea2aed7-5117-49f7-ba5e-ed66fe20b4ac None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:40.459830 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 460/938] 10.222.0.36 () {58 vars in 1439 bytes} [Wed Jul 27 10:35:40 2022] PUT /identity/v3/projects/5f296d8e3b8f4001909addc49d8dac01/users/3d9fed24b4ba44b0b5961a603c4362c4/roles/722097dd0463434b9c4ecaebe7ecf9ea => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:35:40.484276 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 479/939] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 10:35:40 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 27 10:35:40.506874 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea2aed7-5117-49f7-ba5e-ed66fe20b4ac None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:40.558921 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:40.631848 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea2aed7-5117-49f7-ba5e-ed66fe20b4ac None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:40.669717 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1621e6c9-51b3-4c4a-9eee-bc9aabc9beb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:40.670369 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1621e6c9-51b3-4c4a-9eee-bc9aabc9beb9 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:40.670709 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1621e6c9-51b3-4c4a-9eee-bc9aabc9beb9 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:40.670984 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1621e6c9-51b3-4c4a-9eee-bc9aabc9beb9 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:40.671622 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1621e6c9-51b3-4c4a-9eee-bc9aabc9beb9 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:40.673075 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1621e6c9-51b3-4c4a-9eee-bc9aabc9beb9 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:40.682645 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application [None req-1621e6c9-51b3-4c4a-9eee-bc9aabc9beb9 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 27 10:35:40.682645 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:35:40.682645 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 27 10:35:40.682645 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 27 10:35:40.682645 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 27 10:35:40.682645 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 27 10:35:40.682645 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 27 10:35:40.682645 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 10:35:40.682645 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 27 10:35:40.682645 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Jul 27 10:35:40.682645 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 10:35:40.682645 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Jul 27 10:35:40.682645 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:35:40.682645 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:35:40.682645 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:35:40.682645 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:35:40.682645 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:35:40.683663 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:35:40.683663 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:35:40.683663 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:35:40.683663 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:35:40.683663 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:35:40.683663 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:35:40.683663 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 27 10:35:40.683663 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 27 10:35:40.683663 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 27 10:35:40.683663 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 27 10:35:40.683663 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:35:40.683663 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:35:40.683663 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:35:40.683663 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:35:40.683663 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:35:40.683663 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:35:40.684498 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:35:40.684498 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application with Lock( Jul 27 10:35:40.684498 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:35:40.684498 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:35:40.684498 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:35:40.684498 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:35:40.684498 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:35:40.684498 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:35:40.684498 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:35:40.684498 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:35:40.684498 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 27 10:35:40.684498 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 10:35:40.684498 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 27 10:35:40.684498 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Jul 27 10:35:40.684498 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 461/940] 10.222.0.36 () {56 vars in 1126 bytes} [Wed Jul 27 10:35:40 2022] GET /identity/v3/services/object-store => generated 93 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 10:35:40.692092 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193bde4a-a479-40d4-baa8-90daeaaff88b None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:40.771047 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:40.804203 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193bde4a-a479-40d4-baa8-90daeaaff88b None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:40.841709 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ba9c54-9acf-4dfc-a4a5-7fddb6f8a576 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:40.842257 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ba9c54-9acf-4dfc-a4a5-7fddb6f8a576 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:40.842532 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ba9c54-9acf-4dfc-a4a5-7fddb6f8a576 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:40.842833 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ba9c54-9acf-4dfc-a4a5-7fddb6f8a576 None admin] PATH_INFO: `/v3/services` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:40.843518 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ba9c54-9acf-4dfc-a4a5-7fddb6f8a576 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=86827) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:35:40.843840 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ba9c54-9acf-4dfc-a4a5-7fddb6f8a576 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:40.845194 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ba9c54-9acf-4dfc-a4a5-7fddb6f8a576 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:40.850802 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 480/941] 10.222.0.36 () {56 vars in 1122 bytes} [Wed Jul 27 10:35:40 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:35:40.859677 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1621e6c9-51b3-4c4a-9eee-bc9aabc9beb9 None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:40.980928 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1621e6c9-51b3-4c4a-9eee-bc9aabc9beb9 None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:40.990323 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:41.018341 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd476c26-7fcd-46ac-9853-68188b3fe72e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:41.019005 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd476c26-7fcd-46ac-9853-68188b3fe72e None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:41.019291 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd476c26-7fcd-46ac-9853-68188b3fe72e None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:41.019569 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd476c26-7fcd-46ac-9853-68188b3fe72e None admin] PATH_INFO: `/v3/services` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:41.020255 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd476c26-7fcd-46ac-9853-68188b3fe72e None admin] RBAC: Adding query filter params (type=object-store) {{(pid=86828) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:35:41.020562 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd476c26-7fcd-46ac-9853-68188b3fe72e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:41.022501 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd476c26-7fcd-46ac-9853-68188b3fe72e None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:41.028047 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 462/942] 10.222.0.36 () {56 vars in 1122 bytes} [Wed Jul 27 10:35:40 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:35:41.033928 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ba9c54-9acf-4dfc-a4a5-7fddb6f8a576 None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:41.069958 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66da2cf-9d79-40fc-9629-2a9b388ae8ec None None] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:41.070256 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66da2cf-9d79-40fc-9629-2a9b388ae8ec None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:41.070537 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66da2cf-9d79-40fc-9629-2a9b388ae8ec None None] PATH_INFO: `/` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:41.072075 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 463/943] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 10:35:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 10:35:41.077396 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e28dddf-cf85-4fab-bb11-12fca9ae229d None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:41.077687 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e28dddf-cf85-4fab-bb11-12fca9ae229d None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:41.077968 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e28dddf-cf85-4fab-bb11-12fca9ae229d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:41.097122 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-1e28dddf-cf85-4fab-bb11-12fca9ae229d None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:35:41.098251 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-1e28dddf-cf85-4fab-bb11-12fca9ae229d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:35:41.145384 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ba9c54-9acf-4dfc-a4a5-7fddb6f8a576 None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:41.183805 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e3b3ef-663d-4a6b-bafd-595c7e484db4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:41.184431 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e3b3ef-663d-4a6b-bafd-595c7e484db4 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:41.184718 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e3b3ef-663d-4a6b-bafd-595c7e484db4 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:41.184996 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e3b3ef-663d-4a6b-bafd-595c7e484db4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:41.185648 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 464/944] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 10:35:41 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 27 10:35:41.185793 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e3b3ef-663d-4a6b-bafd-595c7e484db4 None admin] RBAC: Adding query filter params (interface=public, service_id=b4a3d610e21d47a8a087acff131a44cd, region_id=RegionOne) {{(pid=86827) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:35:41.186112 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e3b3ef-663d-4a6b-bafd-595c7e484db4 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:41.187574 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e3b3ef-663d-4a6b-bafd-595c7e484db4 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:41.191067 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:41.196059 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 481/945] 10.222.0.36 () {56 vars in 1251 bytes} [Wed Jul 27 10:35:41 2022] GET /identity/v3/endpoints?service_id=b4a3d610e21d47a8a087acff131a44cd&interface=public®ion_id=RegionOne => generated 194 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:35:41.201219 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e28dddf-cf85-4fab-bb11-12fca9ae229d None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:41.211266 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e3b3ef-663d-4a6b-bafd-595c7e484db4 None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:41.305753 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:41.308718 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e28dddf-cf85-4fab-bb11-12fca9ae229d None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:41.336146 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e3b3ef-663d-4a6b-bafd-595c7e484db4 None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:41.349332 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8152284b-d873-4f3d-a8eb-78970ce454bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:41.350249 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8152284b-d873-4f3d-a8eb-78970ce454bd None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:41.350774 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8152284b-d873-4f3d-a8eb-78970ce454bd None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:41.351366 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8152284b-d873-4f3d-a8eb-78970ce454bd None admin] PATH_INFO: `/v3/services` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:41.352393 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8152284b-d873-4f3d-a8eb-78970ce454bd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:41.353966 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8152284b-d873-4f3d-a8eb-78970ce454bd None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:41.360996 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 465/946] 10.222.0.36 () {56 vars in 1087 bytes} [Wed Jul 27 10:35:41 2022] GET /identity/v3/services => generated 1497 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 10:35:41.375160 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23610b4-2ab2-4541-af1f-67854a12de6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:41.376178 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23610b4-2ab2-4541-af1f-67854a12de6c None admin] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:41.376694 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23610b4-2ab2-4541-af1f-67854a12de6c None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:41.377174 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23610b4-2ab2-4541-af1f-67854a12de6c None admin] PATH_INFO: `/v3/projects` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:41.378108 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23610b4-2ab2-4541-af1f-67854a12de6c None admin] RBAC: Authorizing `identity:create_project()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:41.379876 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23610b4-2ab2-4541-af1f-67854a12de6c None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:41.383827 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: WARNING keystone.server.flask.common [None req-c23610b4-2ab2-4541-af1f-67854a12de6c None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jul 27 10:35:41.405256 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.sql.core [None req-c23610b4-2ab2-4541-af1f-67854a12de6c None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 27 10:35:41.405256 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 27 10:35:41.405256 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [parameters: {'id': '48e592bff1074d1d80d1c4109916f53e', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 27 10:35:41.405256 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=86827) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 27 10:35:41.414368 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application [None req-c23610b4-2ab2-4541-af1f-67854a12de6c None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 48e592bff1074d1d80d1c4109916f53e.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 48e592bff1074d1d80d1c4109916f53e. Jul 27 10:35:41.414368 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:35:41.414368 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 27 10:35:41.414368 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 27 10:35:41.414368 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 27 10:35:41.414368 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 27 10:35:41.414368 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 27 10:35:41.414368 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application result = self._query(query) Jul 27 10:35:41.414368 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 27 10:35:41.414368 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application conn.query(q) Jul 27 10:35:41.414368 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 27 10:35:41.414368 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 27 10:35:41.414368 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 27 10:35:41.415298 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application result.read() Jul 27 10:35:41.415298 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 27 10:35:41.415298 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 27 10:35:41.415298 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 27 10:35:41.415298 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 27 10:35:41.415298 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 27 10:35:41.415298 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 27 10:35:41.415298 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 27 10:35:41.415298 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 27 10:35:41.415298 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 27 10:35:41.415298 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Jul 27 10:35:41.415298 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 27 10:35:41.415298 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Jul 27 10:35:41.415298 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:35:41.415298 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 27 10:35:41.415298 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 27 10:35:41.415298 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jul 27 10:35:41.415298 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return project_ref.to_dict() Jul 27 10:35:41.416260 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 27 10:35:41.416260 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application next(self.gen) Jul 27 10:35:41.416260 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 27 10:35:41.416260 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application yield resource Jul 27 10:35:41.416260 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 27 10:35:41.416260 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application next(self.gen) Jul 27 10:35:41.416260 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 27 10:35:41.416260 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application self.session.rollback() Jul 27 10:35:41.416260 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 27 10:35:41.416260 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application self.force_reraise() Jul 27 10:35:41.416260 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 27 10:35:41.416260 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application raise self.value Jul 27 10:35:41.416260 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 27 10:35:41.416260 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 27 10:35:41.416260 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 27 10:35:41.416260 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application session.commit() Jul 27 10:35:41.416260 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 27 10:35:41.417138 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 27 10:35:41.417138 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 27 10:35:41.417138 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application self._prepare_impl() Jul 27 10:35:41.417138 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 27 10:35:41.417138 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application self.session.flush() Jul 27 10:35:41.417138 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 27 10:35:41.417138 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application self._flush(objects) Jul 27 10:35:41.417138 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 27 10:35:41.417138 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 27 10:35:41.417138 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 27 10:35:41.417138 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application compat.raise_( Jul 27 10:35:41.417138 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 27 10:35:41.417138 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application raise exception Jul 27 10:35:41.417138 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 27 10:35:41.417138 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application flush_context.execute() Jul 27 10:35:41.417138 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 27 10:35:41.417138 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application rec.execute(self) Jul 27 10:35:41.418868 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 27 10:35:41.418868 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 27 10:35:41.418868 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 27 10:35:41.418868 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 27 10:35:41.418868 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 27 10:35:41.418868 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 27 10:35:41.418868 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 27 10:35:41.418868 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 27 10:35:41.418868 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 27 10:35:41.418868 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 27 10:35:41.418868 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 27 10:35:41.418868 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 27 10:35:41.418868 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 27 10:35:41.418868 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 27 10:35:41.418868 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 27 10:35:41.420767 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 27 10:35:41.420767 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 27 10:35:41.420767 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application raise exception Jul 27 10:35:41.420767 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 27 10:35:41.420767 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 27 10:35:41.420767 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 27 10:35:41.420767 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 27 10:35:41.420767 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 27 10:35:41.420767 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application result = self._query(query) Jul 27 10:35:41.420767 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 27 10:35:41.420767 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application conn.query(q) Jul 27 10:35:41.420767 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 27 10:35:41.420767 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 27 10:35:41.420767 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 27 10:35:41.420767 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application result.read() Jul 27 10:35:41.420767 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 27 10:35:41.420767 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 27 10:35:41.422953 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 27 10:35:41.422953 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 27 10:35:41.422953 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 27 10:35:41.422953 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 27 10:35:41.422953 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 27 10:35:41.422953 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 27 10:35:41.422953 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 27 10:35:41.422953 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 27 10:35:41.422953 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application [parameters: {'id': '48e592bff1074d1d80d1c4109916f53e', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 27 10:35:41.422953 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 27 10:35:41.422953 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Jul 27 10:35:41.422953 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 10:35:41.422953 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Jul 27 10:35:41.422953 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:35:41.422953 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jul 27 10:35:41.424825 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jul 27 10:35:41.424825 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 27 10:35:41.424825 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 27 10:35:41.424825 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jul 27 10:35:41.424825 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Jul 27 10:35:41.424825 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 10:35:41.424825 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Jul 27 10:35:41.424825 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:35:41.424825 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:35:41.424825 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:35:41.424825 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:35:41.424825 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:35:41.424825 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:35:41.424825 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:35:41.424825 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:35:41.424825 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:35:41.424825 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:35:41.427108 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:35:41.427108 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jul 27 10:35:41.427108 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jul 27 10:35:41.427108 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:35:41.427108 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:35:41.427108 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jul 27 10:35:41.427108 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application raise exception.Conflict( Jul 27 10:35:41.427108 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 48e592bff1074d1d80d1c4109916f53e. Jul 27 10:35:41.427108 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Jul 27 10:35:41.429992 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 482/947] 10.222.0.36 () {60 vars in 1140 bytes} [Wed Jul 27 10:35:41 2022] POST /identity/v3/projects => generated 262 bytes in 219 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jul 27 10:35:41.436933 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8152284b-d873-4f3d-a8eb-78970ce454bd None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:41.556867 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8152284b-d873-4f3d-a8eb-78970ce454bd None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:41.594266 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365a8fbc-64af-47f7-a91f-fc69a1928e7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:41.595244 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365a8fbc-64af-47f7-a91f-fc69a1928e7e None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:41.595532 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365a8fbc-64af-47f7-a91f-fc69a1928e7e None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:41.595863 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365a8fbc-64af-47f7-a91f-fc69a1928e7e None admin] PATH_INFO: `/v3/projects/service` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:41.603507 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365a8fbc-64af-47f7-a91f-fc69a1928e7e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:41.604909 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365a8fbc-64af-47f7-a91f-fc69a1928e7e None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:41.617321 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application [None req-365a8fbc-64af-47f7-a91f-fc69a1928e7e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 10:35:41.617321 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:35:41.617321 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:35:41.617321 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:35:41.617321 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:35:41.617321 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:35:41.617321 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:35:41.617321 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:35:41.617321 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:35:41.617321 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:35:41.617321 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:35:41.617321 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:35:41.617321 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 10:35:41.617321 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 10:35:41.617321 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 10:35:41.618236 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 10:35:41.618236 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:35:41.618236 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:35:41.618236 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:35:41.618236 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:35:41.618236 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:35:41.618236 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:35:41.618236 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:35:41.618236 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application with Lock( Jul 27 10:35:41.618236 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:35:41.618236 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:35:41.618236 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:35:41.618236 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:35:41.618236 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:35:41.618236 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:35:41.618236 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:35:41.618236 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:35:41.619160 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 10:35:41.619160 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 10:35:41.619160 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 10:35:41.619160 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 10:35:41.619160 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 10:35:41.619160 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 10:35:41.619160 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 10:35:41.619160 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Jul 27 10:35:41.619160 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 466/948] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Jul 27 10:35:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 10:35:41.625808 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23610b4-2ab2-4541-af1f-67854a12de6c None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:41.688446 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:41.736551 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23610b4-2ab2-4541-af1f-67854a12de6c None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:41.776671 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a315967-ebcb-40e6-954a-8b6ba52baeb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:41.777649 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a315967-ebcb-40e6-954a-8b6ba52baeb1 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:41.778094 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a315967-ebcb-40e6-954a-8b6ba52baeb1 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:41.778528 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a315967-ebcb-40e6-954a-8b6ba52baeb1 None admin] PATH_INFO: `/v3/projects` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:41.779444 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a315967-ebcb-40e6-954a-8b6ba52baeb1 None admin] RBAC: Adding query filter params (name=service) {{(pid=86827) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:35:41.779910 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a315967-ebcb-40e6-954a-8b6ba52baeb1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:41.781719 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a315967-ebcb-40e6-954a-8b6ba52baeb1 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:41.802528 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 483/949] 10.222.0.36 () {56 vars in 1112 bytes} [Wed Jul 27 10:35:41 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:35:42.429401 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9820fa-7204-4778-b1a0-5298b365bdc3 None None] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:42.429955 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9820fa-7204-4778-b1a0-5298b365bdc3 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:42.429955 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9820fa-7204-4778-b1a0-5298b365bdc3 None None] PATH_INFO: `/` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:42.431594 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 467/950] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 10:35:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 10:35:42.436841 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d948583-4b39-40a0-b264-3a5c1ef8b97a None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:42.437635 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d948583-4b39-40a0-b264-3a5c1ef8b97a None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:42.438128 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d948583-4b39-40a0-b264-3a5c1ef8b97a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:42.456622 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-8d948583-4b39-40a0-b264-3a5c1ef8b97a None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:35:42.458449 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-8d948583-4b39-40a0-b264-3a5c1ef8b97a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:35:42.529812 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 484/951] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 10:35:42 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 10:35:42.551606 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9820fa-7204-4778-b1a0-5298b365bdc3 None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:42.580799 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:42.610453 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aab6551-4ada-4a05-a6bb-c233d63df724 None None] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:42.610819 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aab6551-4ada-4a05-a6bb-c233d63df724 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:42.611076 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aab6551-4ada-4a05-a6bb-c233d63df724 None None] PATH_INFO: `/` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:42.612814 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 485/952] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 10:35:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 10:35:42.620647 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383f28c7-8e59-4f26-acca-b5d5061bea9e None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:42.621210 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383f28c7-8e59-4f26-acca-b5d5061bea9e None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:42.621800 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383f28c7-8e59-4f26-acca-b5d5061bea9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:42.645878 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-383f28c7-8e59-4f26-acca-b5d5061bea9e None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:35:42.647417 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-383f28c7-8e59-4f26-acca-b5d5061bea9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:35:42.673473 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9820fa-7204-4778-b1a0-5298b365bdc3 None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:42.692084 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:42.713413 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c802a3d-7719-4fae-a71e-2c1e2b662ecb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:42.714189 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c802a3d-7719-4fae-a71e-2c1e2b662ecb None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:42.714850 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c802a3d-7719-4fae-a71e-2c1e2b662ecb None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:42.715229 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c802a3d-7719-4fae-a71e-2c1e2b662ecb None admin] PATH_INFO: `/v3/services/compute` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:42.716370 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c802a3d-7719-4fae-a71e-2c1e2b662ecb None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:42.718360 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c802a3d-7719-4fae-a71e-2c1e2b662ecb None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:42.725492 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 486/953] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 10:35:42 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 27 10:35:42.730066 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application [None req-0c802a3d-7719-4fae-a71e-2c1e2b662ecb None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 27 10:35:42.730066 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:35:42.730066 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 27 10:35:42.730066 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 27 10:35:42.730066 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 27 10:35:42.730066 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 27 10:35:42.730066 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 27 10:35:42.730066 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 10:35:42.730066 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 27 10:35:42.730066 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Jul 27 10:35:42.730066 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 10:35:42.730066 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Jul 27 10:35:42.730066 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:35:42.730066 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:35:42.730066 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:35:42.730066 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:35:42.730066 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:35:42.731192 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:35:42.731192 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:35:42.731192 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:35:42.731192 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:35:42.731192 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:35:42.731192 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:35:42.731192 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 27 10:35:42.731192 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 27 10:35:42.731192 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 27 10:35:42.731192 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 27 10:35:42.731192 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:35:42.731192 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:35:42.731192 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:35:42.731192 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:35:42.731192 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:35:42.731192 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:35:42.732234 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:35:42.732234 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application with Lock( Jul 27 10:35:42.732234 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:35:42.732234 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:35:42.732234 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:35:42.732234 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:35:42.732234 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:35:42.732234 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:35:42.732234 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:35:42.732234 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:35:42.732234 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 27 10:35:42.732234 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 10:35:42.732234 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 27 10:35:42.732234 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Jul 27 10:35:42.739193 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 468/954] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Jul 27 10:35:42 2022] GET /identity/v3/services/compute => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 10:35:42.741104 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383f28c7-8e59-4f26-acca-b5d5061bea9e None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:42.752211 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c802a3d-7719-4fae-a71e-2c1e2b662ecb None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:42.870730 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:42.871984 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383f28c7-8e59-4f26-acca-b5d5061bea9e None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:42.887409 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c802a3d-7719-4fae-a71e-2c1e2b662ecb None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:42.918628 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d6f325-5404-41c7-83d2-72052264d9da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:42.919826 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d6f325-5404-41c7-83d2-72052264d9da None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:42.920277 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d6f325-5404-41c7-83d2-72052264d9da None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:42.920957 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d6f325-5404-41c7-83d2-72052264d9da None admin] PATH_INFO: `/v3/services` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:42.922192 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d6f325-5404-41c7-83d2-72052264d9da None admin] RBAC: Adding query filter params (name=compute) {{(pid=86827) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:35:42.922971 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d6f325-5404-41c7-83d2-72052264d9da None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:42.924390 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d6f325-5404-41c7-83d2-72052264d9da None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:42.937008 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 487/955] 10.222.0.36 () {56 vars in 1112 bytes} [Wed Jul 27 10:35:42 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 10:35:42.937176 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5e2610-ccd6-40eb-a763-6ccb68a85621 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:42.937822 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5e2610-ccd6-40eb-a763-6ccb68a85621 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:42.937822 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5e2610-ccd6-40eb-a763-6ccb68a85621 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:42.938119 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5e2610-ccd6-40eb-a763-6ccb68a85621 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:42.939485 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d6f325-5404-41c7-83d2-72052264d9da None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:42.953856 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5e2610-ccd6-40eb-a763-6ccb68a85621 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 10:35:42.961914 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application [None req-3d5e2610-ccd6-40eb-a763-6ccb68a85621 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 27 10:35:42.961914 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:35:42.961914 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:35:42.961914 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:35:42.961914 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:35:42.961914 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:35:42.961914 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:35:42.961914 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:35:42.961914 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:35:42.961914 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:35:42.961914 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:35:42.961914 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:35:42.961914 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 10:35:42.961914 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 10:35:42.961914 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 10:35:42.961914 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 10:35:42.962965 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 10:35:42.962965 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 10:35:42.962965 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 10:35:42.962965 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 10:35:42.962965 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:35:42.962965 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:35:42.962965 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:35:42.962965 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:35:42.962965 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:35:42.962965 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:35:42.962965 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:35:42.962965 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application with Lock( Jul 27 10:35:42.962965 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:35:42.962965 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:35:42.962965 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:35:42.962965 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:35:42.963909 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:35:42.963909 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:35:42.963909 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:35:42.963909 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:35:42.963909 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 10:35:42.963909 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 10:35:42.963909 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 10:35:42.963909 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 10:35:42.963909 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 10:35:42.963909 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 10:35:42.963909 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 27 10:35:42.963909 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Jul 27 10:35:42.967999 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 469/956] 10.222.0.36 () {56 vars in 1120 bytes} [Wed Jul 27 10:35:42 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 10:35:42.976990 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5e2610-ccd6-40eb-a763-6ccb68a85621 None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:42.993054 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:43.065151 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d6f325-5404-41c7-83d2-72052264d9da None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:43.103780 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77aa0358-c421-4b27-9709-3a6d26ab853d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:43.104252 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77aa0358-c421-4b27-9709-3a6d26ab853d None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:43.104548 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77aa0358-c421-4b27-9709-3a6d26ab853d None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:43.104828 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77aa0358-c421-4b27-9709-3a6d26ab853d None admin] PATH_INFO: `/v3/services` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:43.105517 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77aa0358-c421-4b27-9709-3a6d26ab853d None admin] RBAC: Adding query filter params (type=compute) {{(pid=86827) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:35:43.105825 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77aa0358-c421-4b27-9709-3a6d26ab853d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:43.107237 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77aa0358-c421-4b27-9709-3a6d26ab853d None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:43.112635 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 488/957] 10.222.0.36 () {56 vars in 1112 bytes} [Wed Jul 27 10:35:42 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:35:43.127822 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5e2610-ccd6-40eb-a763-6ccb68a85621 None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:43.168798 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9f3ec6-9387-4ba8-9cae-7d1b4605bd58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:43.169514 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9f3ec6-9387-4ba8-9cae-7d1b4605bd58 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:43.169801 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9f3ec6-9387-4ba8-9cae-7d1b4605bd58 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:43.170135 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9f3ec6-9387-4ba8-9cae-7d1b4605bd58 None admin] PATH_INFO: `/v3/roles` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:43.170926 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9f3ec6-9387-4ba8-9cae-7d1b4605bd58 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=86828) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:35:43.171236 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9f3ec6-9387-4ba8-9cae-7d1b4605bd58 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:43.172653 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9f3ec6-9387-4ba8-9cae-7d1b4605bd58 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:43.183125 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 470/958] 10.222.0.36 () {56 vars in 1115 bytes} [Wed Jul 27 10:35:42 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 10:35:43.189396 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77aa0358-c421-4b27-9709-3a6d26ab853d None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:43.305034 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77aa0358-c421-4b27-9709-3a6d26ab853d None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:43.306810 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:43.354999 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03ba50d-dce6-4459-88e8-b880ef376bb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:43.356106 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03ba50d-dce6-4459-88e8-b880ef376bb4 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:43.356638 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03ba50d-dce6-4459-88e8-b880ef376bb4 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:43.357119 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03ba50d-dce6-4459-88e8-b880ef376bb4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:43.367995 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03ba50d-dce6-4459-88e8-b880ef376bb4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:43.369846 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03ba50d-dce6-4459-88e8-b880ef376bb4 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:43.386414 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application [None req-c03ba50d-dce6-4459-88e8-b880ef376bb4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 10:35:43.386414 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:35:43.386414 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:35:43.386414 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:35:43.386414 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:35:43.386414 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:35:43.386414 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:35:43.386414 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:35:43.386414 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:35:43.386414 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:35:43.386414 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:35:43.386414 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:35:43.386414 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 10:35:43.386414 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 10:35:43.386414 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 10:35:43.387505 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 10:35:43.387505 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:35:43.387505 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:35:43.387505 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:35:43.387505 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:35:43.387505 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:35:43.387505 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:35:43.387505 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:35:43.387505 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application with Lock( Jul 27 10:35:43.387505 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:35:43.387505 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:35:43.387505 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:35:43.387505 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:35:43.387505 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:35:43.387505 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:35:43.387505 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:35:43.387505 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:35:43.388398 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 10:35:43.388398 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 10:35:43.388398 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 10:35:43.388398 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Jul 27 10:35:43.390727 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 489/959] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 10:35:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 10:35:43.397166 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9f3ec6-9387-4ba8-9cae-7d1b4605bd58 None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:43.410961 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:43.491580 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55af7f2e-7394-4774-adde-7b8f8ede51a0 None None] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:43.492661 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55af7f2e-7394-4774-adde-7b8f8ede51a0 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:43.493148 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55af7f2e-7394-4774-adde-7b8f8ede51a0 None None] PATH_INFO: `/` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:43.495695 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 490/960] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 10:35:43 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 10:35:43.502857 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15391a0-056a-49eb-97a4-34037bf16a18 None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:43.503263 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15391a0-056a-49eb-97a4-34037bf16a18 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:43.503652 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15391a0-056a-49eb-97a4-34037bf16a18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:43.512427 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9f3ec6-9387-4ba8-9cae-7d1b4605bd58 None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:43.524760 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-b15391a0-056a-49eb-97a4-34037bf16a18 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:35:43.526435 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-b15391a0-056a-49eb-97a4-34037bf16a18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:35:43.554645 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5056e2d3-c7c5-46f6-8bb6-07157ff576b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:43.555802 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5056e2d3-c7c5-46f6-8bb6-07157ff576b5 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:43.556403 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5056e2d3-c7c5-46f6-8bb6-07157ff576b5 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:43.556828 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5056e2d3-c7c5-46f6-8bb6-07157ff576b5 None admin] PATH_INFO: `/v3/domains` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:43.557683 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5056e2d3-c7c5-46f6-8bb6-07157ff576b5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86828) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:35:43.558238 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5056e2d3-c7c5-46f6-8bb6-07157ff576b5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:43.559885 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5056e2d3-c7c5-46f6-8bb6-07157ff576b5 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:43.573691 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 471/961] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 10:35:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:35:43.579414 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5056e2d3-c7c5-46f6-8bb6-07157ff576b5 None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:43.599110 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 491/962] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 10:35:43 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 27 10:35:43.622047 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15391a0-056a-49eb-97a4-34037bf16a18 None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:43.665380 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:43.708288 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5056e2d3-c7c5-46f6-8bb6-07157ff576b5 None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:43.719045 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:43.729222 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15391a0-056a-49eb-97a4-34037bf16a18 None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:43.754415 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c577408-4648-4cf5-91d2-5ba8d9d1761b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:43.755254 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c577408-4648-4cf5-91d2-5ba8d9d1761b None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:43.755537 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c577408-4648-4cf5-91d2-5ba8d9d1761b None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:43.756703 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c577408-4648-4cf5-91d2-5ba8d9d1761b None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:43.765027 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c577408-4648-4cf5-91d2-5ba8d9d1761b None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:43.766382 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c577408-4648-4cf5-91d2-5ba8d9d1761b None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:43.771008 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e29123-8982-4424-a1e3-56a23125c482 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:43.772439 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e29123-8982-4424-a1e3-56a23125c482 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:43.772932 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e29123-8982-4424-a1e3-56a23125c482 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:43.773727 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e29123-8982-4424-a1e3-56a23125c482 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:43.774640 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e29123-8982-4424-a1e3-56a23125c482 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:43.776230 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e29123-8982-4424-a1e3-56a23125c482 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:43.778836 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application [None req-0c577408-4648-4cf5-91d2-5ba8d9d1761b None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 27 10:35:43.778836 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:35:43.778836 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:35:43.778836 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:35:43.778836 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:35:43.778836 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:35:43.778836 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:35:43.778836 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:35:43.778836 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:35:43.778836 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:35:43.778836 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:35:43.778836 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:35:43.778836 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 10:35:43.778836 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 10:35:43.778836 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 10:35:43.778836 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 10:35:43.779791 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:35:43.779791 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:35:43.779791 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 10:35:43.779791 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 10:35:43.779791 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 10:35:43.779791 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 10:35:43.779791 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:35:43.779791 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:35:43.779791 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:35:43.779791 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:35:43.779791 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:35:43.779791 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application with Lock( Jul 27 10:35:43.779791 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:35:43.779791 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:35:43.779791 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:35:43.779791 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:35:43.779791 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:35:43.779791 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:35:43.780761 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:35:43.780761 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:35:43.780761 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 10:35:43.780761 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 10:35:43.780761 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 10:35:43.780761 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 10:35:43.780761 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 10:35:43.780761 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 10:35:43.780761 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 27 10:35:43.780761 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Jul 27 10:35:43.784845 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 472/963] 10.222.0.36 () {56 vars in 1117 bytes} [Wed Jul 27 10:35:43 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 10:35:43.785998 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c577408-4648-4cf5-91d2-5ba8d9d1761b None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:43.791832 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application [None req-97e29123-8982-4424-a1e3-56a23125c482 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 27 10:35:43.791832 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:35:43.791832 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 27 10:35:43.791832 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 27 10:35:43.791832 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 27 10:35:43.791832 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 27 10:35:43.791832 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 27 10:35:43.791832 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 10:35:43.791832 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 27 10:35:43.791832 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Jul 27 10:35:43.791832 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 10:35:43.791832 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Jul 27 10:35:43.791832 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:35:43.791832 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:35:43.791832 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:35:43.791832 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:35:43.791832 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:35:43.793056 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:35:43.793056 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:35:43.793056 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:35:43.793056 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:35:43.793056 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:35:43.793056 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:35:43.793056 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 27 10:35:43.793056 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 27 10:35:43.793056 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 27 10:35:43.793056 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 27 10:35:43.793056 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:35:43.793056 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:35:43.793056 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:35:43.793056 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:35:43.793056 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:35:43.793056 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:35:43.794086 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:35:43.794086 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application with Lock( Jul 27 10:35:43.794086 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:35:43.794086 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:35:43.794086 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:35:43.794086 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:35:43.794086 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:35:43.794086 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:35:43.794086 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:35:43.794086 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:35:43.794086 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 27 10:35:43.794086 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 10:35:43.794086 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 27 10:35:43.794086 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Jul 27 10:35:43.801967 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 492/964] 10.222.0.36 () {56 vars in 1126 bytes} [Wed Jul 27 10:35:43 2022] GET /identity/v3/services/object-store => generated 93 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 10:35:43.803168 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e29123-8982-4424-a1e3-56a23125c482 None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:43.898300 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c577408-4648-4cf5-91d2-5ba8d9d1761b None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:43.928413 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:43.935993 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b047eebc-c069-4a77-b54e-352bd6e6cda6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:43.939607 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b047eebc-c069-4a77-b54e-352bd6e6cda6 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:43.939607 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b047eebc-c069-4a77-b54e-352bd6e6cda6 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:43.939607 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b047eebc-c069-4a77-b54e-352bd6e6cda6 None admin] PATH_INFO: `/v3/users` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:43.939923 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e29123-8982-4424-a1e3-56a23125c482 None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:43.940014 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b047eebc-c069-4a77-b54e-352bd6e6cda6 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=86828) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:35:43.940014 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b047eebc-c069-4a77-b54e-352bd6e6cda6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:43.940255 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b047eebc-c069-4a77-b54e-352bd6e6cda6 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:43.961730 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 473/965] 10.222.0.36 () {56 vars in 1149 bytes} [Wed Jul 27 10:35:43 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:35:43.963607 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc5beee-7079-4bc6-8b41-778fa8212a34 None None] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:43.963963 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc5beee-7079-4bc6-8b41-778fa8212a34 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:43.964309 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc5beee-7079-4bc6-8b41-778fa8212a34 None None] PATH_INFO: `/` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:43.965834 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 474/966] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 10:35:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 10:35:43.966611 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc5beee-7079-4bc6-8b41-778fa8212a34 None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:43.979907 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5597a4b2-37db-4257-beb1-74c03a48a940 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:43.980532 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5597a4b2-37db-4257-beb1-74c03a48a940 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:43.980817 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5597a4b2-37db-4257-beb1-74c03a48a940 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:43.981090 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5597a4b2-37db-4257-beb1-74c03a48a940 None admin] PATH_INFO: `/v3/services` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:43.981770 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5597a4b2-37db-4257-beb1-74c03a48a940 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=86827) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:35:43.982159 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5597a4b2-37db-4257-beb1-74c03a48a940 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:43.983552 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5597a4b2-37db-4257-beb1-74c03a48a940 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:43.988343 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 493/967] 10.222.0.36 () {56 vars in 1122 bytes} [Wed Jul 27 10:35:43 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:35:43.994112 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ab1ab9-65b6-46e1-953e-091afe430851 None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:43.994403 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ab1ab9-65b6-46e1-953e-091afe430851 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:43.994903 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ab1ab9-65b6-46e1-953e-091afe430851 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:44.012449 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-99ab1ab9-65b6-46e1-953e-091afe430851 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:35:44.013632 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-99ab1ab9-65b6-46e1-953e-091afe430851 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:35:44.022546 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:44.076848 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc5beee-7079-4bc6-8b41-778fa8212a34 None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:44.090705 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 494/968] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 10:35:43 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 10:35:44.090705 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ab1ab9-65b6-46e1-953e-091afe430851 None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:44.121347 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68fdc3c-4cad-4a7e-87dc-0bc50b60a59a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:44.121975 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68fdc3c-4cad-4a7e-87dc-0bc50b60a59a None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:44.122259 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68fdc3c-4cad-4a7e-87dc-0bc50b60a59a None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:44.122538 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68fdc3c-4cad-4a7e-87dc-0bc50b60a59a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:44.183384 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:44.197383 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68fdc3c-4cad-4a7e-87dc-0bc50b60a59a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:44.198991 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68fdc3c-4cad-4a7e-87dc-0bc50b60a59a None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:44.204995 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ab1ab9-65b6-46e1-953e-091afe430851 None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:44.241308 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb6c28a-e04e-4edc-8510-afcae3dffa48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:44.241931 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb6c28a-e04e-4edc-8510-afcae3dffa48 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:44.242218 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb6c28a-e04e-4edc-8510-afcae3dffa48 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:44.242492 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb6c28a-e04e-4edc-8510-afcae3dffa48 None admin] PATH_INFO: `/v3/services` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:44.243206 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb6c28a-e04e-4edc-8510-afcae3dffa48 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=86827) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:35:44.243514 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb6c28a-e04e-4edc-8510-afcae3dffa48 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:44.244874 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb6c28a-e04e-4edc-8510-afcae3dffa48 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:44.249873 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 495/969] 10.222.0.36 () {56 vars in 1122 bytes} [Wed Jul 27 10:35:44 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:35:44.250641 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb6c28a-e04e-4edc-8510-afcae3dffa48 None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:44.264047 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:44.311680 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 475/970] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jul 27 10:35:43 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 27 10:35:44.311680 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68fdc3c-4cad-4a7e-87dc-0bc50b60a59a None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:44.368776 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb6c28a-e04e-4edc-8510-afcae3dffa48 None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:44.404155 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:44.407642 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff5d53f-42b5-4163-9499-a96e4cc6ef9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:44.408451 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff5d53f-42b5-4163-9499-a96e4cc6ef9b None admin] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:44.408734 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff5d53f-42b5-4163-9499-a96e4cc6ef9b None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:44.409011 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff5d53f-42b5-4163-9499-a96e4cc6ef9b None admin] PATH_INFO: `/v3/users` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:44.409692 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff5d53f-42b5-4163-9499-a96e4cc6ef9b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:44.411371 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff5d53f-42b5-4163-9499-a96e4cc6ef9b None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:44.416574 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68fdc3c-4cad-4a7e-87dc-0bc50b60a59a None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:44.425992 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.sql.core [None req-9ff5d53f-42b5-4163-9499-a96e4cc6ef9b None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 27 10:35:44.425992 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 27 10:35:44.425992 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [parameters: {'user_id': '56d11238c3a24d33a670a391849da105', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 27 10:35:44.425992 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=86827) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 27 10:35:44.433330 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application [None req-9ff5d53f-42b5-4163-9499-a96e4cc6ef9b None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 27 10:35:44.433330 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:35:44.433330 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 27 10:35:44.433330 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 27 10:35:44.433330 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 27 10:35:44.433330 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 27 10:35:44.433330 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 27 10:35:44.433330 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application result = self._query(query) Jul 27 10:35:44.433330 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 27 10:35:44.433330 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application conn.query(q) Jul 27 10:35:44.433330 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 27 10:35:44.433330 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 27 10:35:44.433330 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 27 10:35:44.433330 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application result.read() Jul 27 10:35:44.433330 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 27 10:35:44.434281 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 27 10:35:44.434281 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 27 10:35:44.434281 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 27 10:35:44.434281 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 27 10:35:44.434281 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 27 10:35:44.434281 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 27 10:35:44.434281 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 27 10:35:44.434281 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 27 10:35:44.434281 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Jul 27 10:35:44.434281 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 27 10:35:44.434281 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Jul 27 10:35:44.434281 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:35:44.434281 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 27 10:35:44.434281 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 27 10:35:44.434281 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jul 27 10:35:44.434281 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jul 27 10:35:44.434281 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 27 10:35:44.434281 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application next(self.gen) Jul 27 10:35:44.435301 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 27 10:35:44.435301 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application yield resource Jul 27 10:35:44.435301 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 27 10:35:44.435301 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application next(self.gen) Jul 27 10:35:44.435301 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 27 10:35:44.435301 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application self.session.rollback() Jul 27 10:35:44.435301 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 27 10:35:44.435301 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application self.force_reraise() Jul 27 10:35:44.435301 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 27 10:35:44.435301 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application raise self.value Jul 27 10:35:44.435301 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 27 10:35:44.435301 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 27 10:35:44.435301 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 27 10:35:44.435301 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application session.commit() Jul 27 10:35:44.435301 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 27 10:35:44.435301 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 27 10:35:44.435301 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 27 10:35:44.436198 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application self._prepare_impl() Jul 27 10:35:44.436198 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 27 10:35:44.436198 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application self.session.flush() Jul 27 10:35:44.436198 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 27 10:35:44.436198 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application self._flush(objects) Jul 27 10:35:44.436198 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 27 10:35:44.436198 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 27 10:35:44.436198 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 27 10:35:44.436198 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application compat.raise_( Jul 27 10:35:44.436198 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 27 10:35:44.436198 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application raise exception Jul 27 10:35:44.436198 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 27 10:35:44.436198 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application flush_context.execute() Jul 27 10:35:44.436198 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 27 10:35:44.436198 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application rec.execute(self) Jul 27 10:35:44.436198 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 27 10:35:44.436198 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 27 10:35:44.437870 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 27 10:35:44.437870 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 27 10:35:44.437870 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jul 27 10:35:44.437870 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application result = connection._execute_20( Jul 27 10:35:44.437870 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 27 10:35:44.437870 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 27 10:35:44.437870 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 27 10:35:44.437870 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 27 10:35:44.437870 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 27 10:35:44.437870 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 27 10:35:44.437870 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 27 10:35:44.437870 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 27 10:35:44.437870 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 27 10:35:44.437870 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 27 10:35:44.437870 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 27 10:35:44.437870 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application raise exception Jul 27 10:35:44.439797 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 27 10:35:44.439797 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 27 10:35:44.439797 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 27 10:35:44.439797 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 27 10:35:44.439797 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 27 10:35:44.439797 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application result = self._query(query) Jul 27 10:35:44.439797 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 27 10:35:44.439797 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application conn.query(q) Jul 27 10:35:44.439797 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 27 10:35:44.439797 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 27 10:35:44.439797 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 27 10:35:44.439797 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application result.read() Jul 27 10:35:44.439797 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 27 10:35:44.439797 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 27 10:35:44.439797 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 27 10:35:44.439797 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 27 10:35:44.440900 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 27 10:35:44.440900 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 27 10:35:44.440900 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 27 10:35:44.440900 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 27 10:35:44.440900 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 27 10:35:44.440900 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 27 10:35:44.440900 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application [parameters: {'user_id': '56d11238c3a24d33a670a391849da105', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 27 10:35:44.440900 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 27 10:35:44.440900 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Jul 27 10:35:44.440900 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 10:35:44.440900 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Jul 27 10:35:44.440900 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:35:44.440900 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:35:44.440900 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:35:44.440900 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:35:44.441879 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:35:44.441879 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:35:44.441879 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:35:44.441879 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:35:44.441879 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:35:44.441879 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:35:44.441879 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:35:44.441879 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jul 27 10:35:44.441879 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jul 27 10:35:44.441879 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:35:44.441879 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:35:44.441879 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 10:35:44.441879 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 10:35:44.441879 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 10:35:44.441879 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 10:35:44.441879 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jul 27 10:35:44.441879 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jul 27 10:35:44.443011 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jul 27 10:35:44.443011 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jul 27 10:35:44.443011 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 27 10:35:44.443011 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 27 10:35:44.443011 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 27 10:35:44.443011 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Jul 27 10:35:44.443011 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 496/971] 10.222.0.36 () {60 vars in 1131 bytes} [Wed Jul 27 10:35:44 2022] POST /identity/v3/users => generated 160 bytes in 184 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 27 10:35:44.443011 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff5d53f-42b5-4163-9499-a96e4cc6ef9b None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:44.465953 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e41f33e-a4d4-4443-bd39-ca2502bd5219 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:44.466633 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e41f33e-a4d4-4443-bd39-ca2502bd5219 None admin] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:44.466876 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e41f33e-a4d4-4443-bd39-ca2502bd5219 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:44.467302 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e41f33e-a4d4-4443-bd39-ca2502bd5219 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:44.468009 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e41f33e-a4d4-4443-bd39-ca2502bd5219 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:44.469370 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e41f33e-a4d4-4443-bd39-ca2502bd5219 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:44.489774 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 476/972] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Jul 27 10:35:44 2022] POST /identity/v3/endpoints => generated 349 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 10:35:44.499897 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e41f33e-a4d4-4443-bd39-ca2502bd5219 None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:44.518452 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:44.556553 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff5d53f-42b5-4163-9499-a96e4cc6ef9b None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:44.614584 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adfd772-89ee-480e-b00e-94891cfb146f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:44.615886 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adfd772-89ee-480e-b00e-94891cfb146f None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:44.616171 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adfd772-89ee-480e-b00e-94891cfb146f None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:44.616448 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adfd772-89ee-480e-b00e-94891cfb146f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:44.632022 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4adfd772-89ee-480e-b00e-94891cfb146f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:44.634215 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4adfd772-89ee-480e-b00e-94891cfb146f None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:44.657031 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application [None req-4adfd772-89ee-480e-b00e-94891cfb146f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 10:35:44.657031 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:35:44.657031 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:35:44.657031 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:35:44.657031 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:35:44.657031 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:35:44.657031 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:35:44.657031 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:35:44.657031 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:35:44.657031 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:35:44.657031 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:35:44.657031 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:35:44.657031 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 10:35:44.657031 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 10:35:44.657031 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 10:35:44.657983 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 10:35:44.657983 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:35:44.657983 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:35:44.657983 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:35:44.657983 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:35:44.657983 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:35:44.657983 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:35:44.657983 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:35:44.657983 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application with Lock( Jul 27 10:35:44.657983 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:35:44.657983 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:35:44.657983 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:35:44.657983 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:35:44.657983 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:35:44.657983 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:35:44.657983 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:35:44.657983 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:35:44.658933 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 10:35:44.658933 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 10:35:44.658933 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 10:35:44.658933 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Jul 27 10:35:44.658933 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 497/973] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 10:35:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 10:35:44.659216 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e41f33e-a4d4-4443-bd39-ca2502bd5219 None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:44.665809 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adfd772-89ee-480e-b00e-94891cfb146f None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:44.711257 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd31174-b7f0-46dd-a619-86c5636db436 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:44.711835 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd31174-b7f0-46dd-a619-86c5636db436 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:44.712118 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd31174-b7f0-46dd-a619-86c5636db436 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:44.712395 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd31174-b7f0-46dd-a619-86c5636db436 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:44.720510 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd31174-b7f0-46dd-a619-86c5636db436 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:44.722000 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd31174-b7f0-46dd-a619-86c5636db436 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:44.736938 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application [None req-3cd31174-b7f0-46dd-a619-86c5636db436 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 27 10:35:44.736938 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:35:44.736938 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:35:44.736938 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:35:44.736938 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:35:44.736938 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:35:44.736938 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:35:44.736938 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:35:44.736938 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:35:44.736938 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:35:44.736938 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:35:44.736938 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:35:44.736938 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 10:35:44.736938 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 10:35:44.736938 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 10:35:44.736938 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 10:35:44.737917 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:35:44.737917 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:35:44.737917 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 10:35:44.737917 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 10:35:44.737917 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 10:35:44.737917 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 10:35:44.737917 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:35:44.737917 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:35:44.737917 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:35:44.737917 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:35:44.737917 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:35:44.737917 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application with Lock( Jul 27 10:35:44.737917 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:35:44.737917 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:35:44.737917 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:35:44.737917 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:35:44.737917 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:35:44.737917 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:35:44.738882 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:35:44.738882 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:35:44.738882 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 10:35:44.738882 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 10:35:44.738882 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 10:35:44.738882 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 10:35:44.738882 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 10:35:44.738882 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 10:35:44.738882 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 27 10:35:44.738882 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Jul 27 10:35:44.738882 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 477/974] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jul 27 10:35:44 2022] GET /identity/v3/users/cinder => generated 84 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 10:35:44.746938 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd31174-b7f0-46dd-a619-86c5636db436 None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:44.762963 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:44.787108 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adfd772-89ee-480e-b00e-94891cfb146f None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:44.804747 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:44.823936 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca13bb9e-a849-4a1c-8bc2-b1976e25fbba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:44.824564 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca13bb9e-a849-4a1c-8bc2-b1976e25fbba None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:44.824848 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca13bb9e-a849-4a1c-8bc2-b1976e25fbba None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:44.825128 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca13bb9e-a849-4a1c-8bc2-b1976e25fbba None admin] PATH_INFO: `/v3/domains` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:44.825829 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca13bb9e-a849-4a1c-8bc2-b1976e25fbba None admin] RBAC: Adding query filter params (name=Default) {{(pid=86827) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:35:44.826146 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca13bb9e-a849-4a1c-8bc2-b1976e25fbba None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:44.827869 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca13bb9e-a849-4a1c-8bc2-b1976e25fbba None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:44.844662 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 498/975] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 10:35:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:35:44.845651 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca13bb9e-a849-4a1c-8bc2-b1976e25fbba None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:44.865147 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd31174-b7f0-46dd-a619-86c5636db436 None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:44.905797 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee15c56-2d03-43a9-b3fe-7bd3a99a5b4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:44.907157 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee15c56-2d03-43a9-b3fe-7bd3a99a5b4e None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:44.907883 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee15c56-2d03-43a9-b3fe-7bd3a99a5b4e None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:44.908308 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee15c56-2d03-43a9-b3fe-7bd3a99a5b4e None admin] PATH_INFO: `/v3/users` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:44.909480 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee15c56-2d03-43a9-b3fe-7bd3a99a5b4e None admin] RBAC: Adding query filter params (name=cinder) {{(pid=86828) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:35:44.909934 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee15c56-2d03-43a9-b3fe-7bd3a99a5b4e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:44.911743 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee15c56-2d03-43a9-b3fe-7bd3a99a5b4e None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:44.963070 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca13bb9e-a849-4a1c-8bc2-b1976e25fbba None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:44.977786 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 478/976] 10.222.0.36 () {56 vars in 1101 bytes} [Wed Jul 27 10:35:44 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:35:45.001966 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce42bbf1-ec6f-4f72-8c4a-6212120e5e79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:45.002662 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce42bbf1-ec6f-4f72-8c4a-6212120e5e79 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:45.002951 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce42bbf1-ec6f-4f72-8c4a-6212120e5e79 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:45.003228 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce42bbf1-ec6f-4f72-8c4a-6212120e5e79 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:45.011501 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce42bbf1-ec6f-4f72-8c4a-6212120e5e79 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:45.012982 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce42bbf1-ec6f-4f72-8c4a-6212120e5e79 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:45.027607 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application [None req-ce42bbf1-ec6f-4f72-8c4a-6212120e5e79 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 10:35:45.027607 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:35:45.027607 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:35:45.027607 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:35:45.027607 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:35:45.027607 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:35:45.027607 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:35:45.027607 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:35:45.027607 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:35:45.027607 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:35:45.027607 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:35:45.027607 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:35:45.027607 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 10:35:45.027607 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 10:35:45.027607 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 10:35:45.028757 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 10:35:45.028757 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:35:45.028757 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:35:45.028757 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:35:45.028757 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:35:45.028757 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:35:45.028757 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:35:45.028757 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:35:45.028757 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application with Lock( Jul 27 10:35:45.028757 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:35:45.028757 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:35:45.028757 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:35:45.028757 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:35:45.028757 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:35:45.028757 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:35:45.028757 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:35:45.028757 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:35:45.029851 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 10:35:45.029851 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 10:35:45.029851 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 10:35:45.029851 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 10:35:45.029851 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 10:35:45.029851 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 10:35:45.029851 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 10:35:45.029851 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Jul 27 10:35:45.029851 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 499/977] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Jul 27 10:35:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 10:35:45.034950 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee15c56-2d03-43a9-b3fe-7bd3a99a5b4e None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:45.105995 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:45.149035 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee15c56-2d03-43a9-b3fe-7bd3a99a5b4e None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:45.185301 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ee999a-6712-481d-aab7-82256273d253 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:45.185939 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ee999a-6712-481d-aab7-82256273d253 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:45.186229 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ee999a-6712-481d-aab7-82256273d253 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:45.186505 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ee999a-6712-481d-aab7-82256273d253 None admin] PATH_INFO: `/v3/projects` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:45.187298 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ee999a-6712-481d-aab7-82256273d253 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86828) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:35:45.187607 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ee999a-6712-481d-aab7-82256273d253 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:45.189051 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ee999a-6712-481d-aab7-82256273d253 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:45.200928 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f33a61f-45b9-4092-8d69-27b58c260d2f None None] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:45.201219 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f33a61f-45b9-4092-8d69-27b58c260d2f None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:45.201491 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f33a61f-45b9-4092-8d69-27b58c260d2f None None] PATH_INFO: `/` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:45.203143 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 500/978] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 10:35:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 10:35:45.205095 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 479/979] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Jul 27 10:35:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:35:45.209984 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84611619-43a3-4fbb-8bef-3bca2c8b3dc1 None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:45.210263 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84611619-43a3-4fbb-8bef-3bca2c8b3dc1 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:45.210705 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84611619-43a3-4fbb-8bef-3bca2c8b3dc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:45.213654 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ee999a-6712-481d-aab7-82256273d253 None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:45.232923 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-84611619-43a3-4fbb-8bef-3bca2c8b3dc1 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:35:45.234768 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-84611619-43a3-4fbb-8bef-3bca2c8b3dc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:35:45.265592 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:45.314895 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 501/980] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 10:35:45 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 27 10:35:45.340585 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84611619-43a3-4fbb-8bef-3bca2c8b3dc1 None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:45.345155 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ee999a-6712-481d-aab7-82256273d253 None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:45.371970 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:45.385335 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbd160a-eb96-42da-af1b-ffae66d4a7ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:45.385962 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbd160a-eb96-42da-af1b-ffae66d4a7ee None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:45.386247 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbd160a-eb96-42da-af1b-ffae66d4a7ee None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:45.386530 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbd160a-eb96-42da-af1b-ffae66d4a7ee None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:45.387368 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbd160a-eb96-42da-af1b-ffae66d4a7ee None admin] RBAC: Adding query filter params (role.id=722097dd0463434b9c4ecaebe7ecf9ea, scope.project.id=5f296d8e3b8f4001909addc49d8dac01, user.id=3d9fed24b4ba44b0b5961a603c4362c4) {{(pid=86828) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:35:45.387685 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbd160a-eb96-42da-af1b-ffae66d4a7ee None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:45.389126 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbd160a-eb96-42da-af1b-ffae66d4a7ee None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:45.395138 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 480/981] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Jul 27 10:35:45 2022] GET /identity/v3/role_assignments?user.id=3d9fed24b4ba44b0b5961a603c4362c4&scope.project.id=5f296d8e3b8f4001909addc49d8dac01&role.id=722097dd0463434b9c4ecaebe7ecf9ea => generated 608 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:35:45.455706 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84611619-43a3-4fbb-8bef-3bca2c8b3dc1 None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:45.494585 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99276d9-d1d0-44c7-9689-ff6dd15596d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:45.495513 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99276d9-d1d0-44c7-9689-ff6dd15596d3 None admin] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:45.495942 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99276d9-d1d0-44c7-9689-ff6dd15596d3 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:45.496371 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99276d9-d1d0-44c7-9689-ff6dd15596d3 None admin] PATH_INFO: `/v3/services` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:45.497230 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99276d9-d1d0-44c7-9689-ff6dd15596d3 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:45.498797 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99276d9-d1d0-44c7-9689-ff6dd15596d3 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:45.515581 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 502/982] 10.222.0.36 () {60 vars in 1141 bytes} [Wed Jul 27 10:35:45 2022] POST /identity/v3/services => generated 242 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 10:35:46.596886 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f7a1d5-208c-4bf9-b426-29fe9507a529 None None] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:46.597397 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f7a1d5-208c-4bf9-b426-29fe9507a529 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:46.597475 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f7a1d5-208c-4bf9-b426-29fe9507a529 None None] PATH_INFO: `/` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:46.598958 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 481/983] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 10:35:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 10:35:46.604678 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f748ff31-23c7-40c6-bf2f-1af467cbb371 None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:46.605097 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f748ff31-23c7-40c6-bf2f-1af467cbb371 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:46.605231 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f748ff31-23c7-40c6-bf2f-1af467cbb371 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:46.623934 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-f748ff31-23c7-40c6-bf2f-1af467cbb371 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:35:46.625172 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-f748ff31-23c7-40c6-bf2f-1af467cbb371 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:35:46.648858 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:46.706124 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 503/984] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 10:35:46 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 10:35:46.728781 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f7a1d5-208c-4bf9-b426-29fe9507a529 None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:46.830934 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f7a1d5-208c-4bf9-b426-29fe9507a529 None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:46.864139 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2bd88d-9b68-4dd7-af60-541ed0c4bf82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:46.864703 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2bd88d-9b68-4dd7-af60-541ed0c4bf82 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:46.865001 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2bd88d-9b68-4dd7-af60-541ed0c4bf82 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:46.865277 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2bd88d-9b68-4dd7-af60-541ed0c4bf82 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:46.865895 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2bd88d-9b68-4dd7-af60-541ed0c4bf82 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:46.867274 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2bd88d-9b68-4dd7-af60-541ed0c4bf82 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:46.876299 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application [None req-8d2bd88d-9b68-4dd7-af60-541ed0c4bf82 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 27 10:35:46.876299 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:35:46.876299 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 27 10:35:46.876299 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 27 10:35:46.876299 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 27 10:35:46.876299 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 27 10:35:46.876299 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 27 10:35:46.876299 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 10:35:46.876299 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 27 10:35:46.876299 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Jul 27 10:35:46.876299 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 10:35:46.876299 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Jul 27 10:35:46.876299 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:35:46.876299 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:35:46.876299 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:35:46.876299 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:35:46.876299 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:35:46.877275 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:35:46.877275 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:35:46.877275 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:35:46.877275 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:35:46.877275 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:35:46.877275 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:35:46.877275 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 27 10:35:46.877275 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 27 10:35:46.877275 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 27 10:35:46.877275 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 27 10:35:46.877275 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:35:46.877275 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:35:46.877275 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:35:46.877275 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:35:46.877275 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:35:46.877275 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:35:46.878129 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:35:46.878129 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application with Lock( Jul 27 10:35:46.878129 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:35:46.878129 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:35:46.878129 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:35:46.878129 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:35:46.878129 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:35:46.878129 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:35:46.878129 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:35:46.878129 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:35:46.878129 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 27 10:35:46.878129 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 10:35:46.878129 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 27 10:35:46.878129 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Jul 27 10:35:46.882451 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 482/985] 10.222.0.36 () {56 vars in 1126 bytes} [Wed Jul 27 10:35:46 2022] GET /identity/v3/services/object-store => generated 93 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 10:35:46.883976 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f748ff31-23c7-40c6-bf2f-1af467cbb371 None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:46.994229 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f748ff31-23c7-40c6-bf2f-1af467cbb371 None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:47.029861 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83dd0014-bc7b-4e23-954f-36fbb7c5260e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:47.030485 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dd0014-bc7b-4e23-954f-36fbb7c5260e None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:47.030788 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dd0014-bc7b-4e23-954f-36fbb7c5260e None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:47.031112 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dd0014-bc7b-4e23-954f-36fbb7c5260e None admin] PATH_INFO: `/v3/services` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:47.031732 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83dd0014-bc7b-4e23-954f-36fbb7c5260e None admin] RBAC: Adding query filter params (name=object-store) {{(pid=86827) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:35:47.032031 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83dd0014-bc7b-4e23-954f-36fbb7c5260e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:47.033462 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83dd0014-bc7b-4e23-954f-36fbb7c5260e None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:47.043495 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 504/986] 10.222.0.36 () {56 vars in 1122 bytes} [Wed Jul 27 10:35:46 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:35:47.045010 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2bd88d-9b68-4dd7-af60-541ed0c4bf82 None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:47.097016 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:47.148889 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2bd88d-9b68-4dd7-af60-541ed0c4bf82 None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:47.182974 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69148d4f-5090-415a-acb4-e2a7ef903d40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:47.183596 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69148d4f-5090-415a-acb4-e2a7ef903d40 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:47.183876 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69148d4f-5090-415a-acb4-e2a7ef903d40 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:47.184147 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69148d4f-5090-415a-acb4-e2a7ef903d40 None admin] PATH_INFO: `/v3/services` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:47.184817 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69148d4f-5090-415a-acb4-e2a7ef903d40 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=86828) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:35:47.185119 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69148d4f-5090-415a-acb4-e2a7ef903d40 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:47.186470 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69148d4f-5090-415a-acb4-e2a7ef903d40 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:47.191483 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 483/987] 10.222.0.36 () {56 vars in 1122 bytes} [Wed Jul 27 10:35:47 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:35:47.197763 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83dd0014-bc7b-4e23-954f-36fbb7c5260e None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:47.216543 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:47.298029 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83dd0014-bc7b-4e23-954f-36fbb7c5260e None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:47.331838 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9496fe-a1d8-4f0b-a2da-f1f09d2febf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:47.332596 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9496fe-a1d8-4f0b-a2da-f1f09d2febf9 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:47.332972 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9496fe-a1d8-4f0b-a2da-f1f09d2febf9 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:47.333336 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9496fe-a1d8-4f0b-a2da-f1f09d2febf9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:47.334845 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9496fe-a1d8-4f0b-a2da-f1f09d2febf9 None admin] RBAC: Adding query filter params (interface=admin, service_id=b4a3d610e21d47a8a087acff131a44cd, region_id=RegionOne) {{(pid=86827) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:35:47.335252 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9496fe-a1d8-4f0b-a2da-f1f09d2febf9 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:47.336715 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9496fe-a1d8-4f0b-a2da-f1f09d2febf9 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:47.342026 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 505/988] 10.222.0.36 () {56 vars in 1249 bytes} [Wed Jul 27 10:35:47 2022] GET /identity/v3/endpoints?service_id=b4a3d610e21d47a8a087acff131a44cd&interface=admin®ion_id=RegionOne => generated 193 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:35:47.347487 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69148d4f-5090-415a-acb4-e2a7ef903d40 None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:47.439515 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f7f963-dd71-46d1-ab51-42903d30541a None None] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:47.439811 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f7f963-dd71-46d1-ab51-42903d30541a None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:47.440071 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f7f963-dd71-46d1-ab51-42903d30541a None None] PATH_INFO: `/` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:47.441487 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 506/989] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 10:35:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 10:35:47.446761 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38713f02-6fe1-4a0b-8c05-85d1548ec468 None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:47.447149 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38713f02-6fe1-4a0b-8c05-85d1548ec468 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:47.447522 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38713f02-6fe1-4a0b-8c05-85d1548ec468 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:47.447627 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69148d4f-5090-415a-acb4-e2a7ef903d40 None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:47.463287 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:47.466366 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-38713f02-6fe1-4a0b-8c05-85d1548ec468 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:35:47.467623 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-38713f02-6fe1-4a0b-8c05-85d1548ec468 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:35:47.485643 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65f3dc8-2f68-48fd-9af5-bf24a37e5611 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:47.486357 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65f3dc8-2f68-48fd-9af5-bf24a37e5611 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:47.486750 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65f3dc8-2f68-48fd-9af5-bf24a37e5611 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:47.487103 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65f3dc8-2f68-48fd-9af5-bf24a37e5611 None admin] PATH_INFO: `/v3/services` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:47.487833 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65f3dc8-2f68-48fd-9af5-bf24a37e5611 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:47.489278 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65f3dc8-2f68-48fd-9af5-bf24a37e5611 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:47.496403 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 484/990] 10.222.0.36 () {56 vars in 1087 bytes} [Wed Jul 27 10:35:47 2022] GET /identity/v3/services => generated 1727 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 10:35:47.539404 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 507/991] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 10:35:47 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 27 10:35:47.547741 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caaccd85-b14c-48fa-9e69-f7d9d1a52daf None None] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:47.548074 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caaccd85-b14c-48fa-9e69-f7d9d1a52daf None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:47.548311 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caaccd85-b14c-48fa-9e69-f7d9d1a52daf None None] PATH_INFO: `/` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:47.549895 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 485/992] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 10:35:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 10:35:47.556075 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8926bffc-2761-4e9a-8dae-27282808abab None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:47.556364 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8926bffc-2761-4e9a-8dae-27282808abab None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:47.556639 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8926bffc-2761-4e9a-8dae-27282808abab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:47.563443 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caaccd85-b14c-48fa-9e69-f7d9d1a52daf None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:47.576657 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-8926bffc-2761-4e9a-8dae-27282808abab None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:35:47.577979 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-8926bffc-2761-4e9a-8dae-27282808abab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:35:47.581266 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:47.664760 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 508/993] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 10:35:47 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 10:35:47.686857 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caaccd85-b14c-48fa-9e69-f7d9d1a52daf None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:47.687725 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8926bffc-2761-4e9a-8dae-27282808abab None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:47.723894 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abc8b7f-c986-4944-af01-1204467e7e39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:47.724492 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abc8b7f-c986-4944-af01-1204467e7e39 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:47.724783 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abc8b7f-c986-4944-af01-1204467e7e39 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:47.725084 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abc8b7f-c986-4944-af01-1204467e7e39 None admin] PATH_INFO: `/v3/services/image` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:47.725772 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7abc8b7f-c986-4944-af01-1204467e7e39 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:47.727300 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7abc8b7f-c986-4944-af01-1204467e7e39 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:47.736292 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application [None req-7abc8b7f-c986-4944-af01-1204467e7e39 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 27 10:35:47.736292 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:35:47.736292 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 27 10:35:47.736292 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 27 10:35:47.736292 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 27 10:35:47.736292 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 27 10:35:47.736292 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 27 10:35:47.736292 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 10:35:47.736292 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 27 10:35:47.736292 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Jul 27 10:35:47.736292 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 10:35:47.736292 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Jul 27 10:35:47.736292 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:35:47.736292 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:35:47.736292 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:35:47.736292 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:35:47.736292 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:35:47.738682 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:35:47.738682 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:35:47.738682 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:35:47.738682 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:35:47.738682 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:35:47.738682 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:35:47.738682 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 27 10:35:47.738682 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 27 10:35:47.738682 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 27 10:35:47.738682 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 27 10:35:47.738682 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:35:47.738682 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:35:47.738682 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:35:47.738682 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:35:47.738682 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:35:47.738682 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:35:47.739673 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:35:47.739673 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application with Lock( Jul 27 10:35:47.739673 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:35:47.739673 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:35:47.739673 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:35:47.739673 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:35:47.739673 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:35:47.739673 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:35:47.739673 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:35:47.739673 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:35:47.739673 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 27 10:35:47.739673 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 10:35:47.739673 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 27 10:35:47.739673 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Jul 27 10:35:47.739673 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 486/994] 10.222.0.36 () {56 vars in 1105 bytes} [Wed Jul 27 10:35:47 2022] GET /identity/v3/services/image => generated 86 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 10:35:47.744015 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abc8b7f-c986-4944-af01-1204467e7e39 None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:47.792881 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:47.803402 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8926bffc-2761-4e9a-8dae-27282808abab None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:47.838357 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b58ee0-082e-4004-81ce-eb4b3d6f1777 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:47.839052 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b58ee0-082e-4004-81ce-eb4b3d6f1777 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:47.839336 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b58ee0-082e-4004-81ce-eb4b3d6f1777 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:47.839614 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b58ee0-082e-4004-81ce-eb4b3d6f1777 None admin] PATH_INFO: `/v3/services/compute` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:47.840249 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b58ee0-082e-4004-81ce-eb4b3d6f1777 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:47.841628 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b58ee0-082e-4004-81ce-eb4b3d6f1777 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:47.851103 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application [None req-54b58ee0-082e-4004-81ce-eb4b3d6f1777 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 27 10:35:47.851103 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:35:47.851103 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 27 10:35:47.851103 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 27 10:35:47.851103 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 27 10:35:47.851103 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 27 10:35:47.851103 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 27 10:35:47.851103 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 10:35:47.851103 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 27 10:35:47.851103 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Jul 27 10:35:47.851103 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 10:35:47.851103 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Jul 27 10:35:47.851103 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:35:47.851103 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:35:47.851103 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:35:47.851103 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:35:47.851103 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:35:47.852072 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:35:47.852072 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:35:47.852072 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:35:47.852072 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:35:47.852072 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:35:47.852072 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:35:47.852072 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 27 10:35:47.852072 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 27 10:35:47.852072 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 27 10:35:47.852072 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 27 10:35:47.852072 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:35:47.852072 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:35:47.852072 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:35:47.852072 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:35:47.852072 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:35:47.852072 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:35:47.853602 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:35:47.853602 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application with Lock( Jul 27 10:35:47.853602 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:35:47.853602 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:35:47.853602 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:35:47.853602 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:35:47.853602 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:35:47.853602 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:35:47.853602 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:35:47.853602 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:35:47.853602 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 27 10:35:47.853602 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 10:35:47.853602 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 27 10:35:47.853602 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Jul 27 10:35:47.853602 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 509/995] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Jul 27 10:35:47 2022] GET /identity/v3/services/compute => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 10:35:47.856861 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abc8b7f-c986-4944-af01-1204467e7e39 None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:47.858051 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b58ee0-082e-4004-81ce-eb4b3d6f1777 None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:47.861848 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:47.896353 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2cc869-b87d-4dcd-b96a-cd3953669763 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:47.896986 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2cc869-b87d-4dcd-b96a-cd3953669763 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:47.897269 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2cc869-b87d-4dcd-b96a-cd3953669763 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:47.897542 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2cc869-b87d-4dcd-b96a-cd3953669763 None admin] PATH_INFO: `/v3/services` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:47.898229 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2cc869-b87d-4dcd-b96a-cd3953669763 None admin] RBAC: Adding query filter params (name=image) {{(pid=86828) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:35:47.898539 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2cc869-b87d-4dcd-b96a-cd3953669763 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:47.899997 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2cc869-b87d-4dcd-b96a-cd3953669763 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:47.905273 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 487/996] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 10:35:47 2022] GET /identity/v3/services?name=image => generated 122 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 10:35:47.910850 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2cc869-b87d-4dcd-b96a-cd3953669763 None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:47.969838 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b58ee0-082e-4004-81ce-eb4b3d6f1777 None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:48.005589 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4cf3ce-9b7e-4099-89e5-40e16bb7b40d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:48.006438 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4cf3ce-9b7e-4099-89e5-40e16bb7b40d None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:48.006912 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4cf3ce-9b7e-4099-89e5-40e16bb7b40d None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:48.007367 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4cf3ce-9b7e-4099-89e5-40e16bb7b40d None admin] PATH_INFO: `/v3/services` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:48.008198 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4cf3ce-9b7e-4099-89e5-40e16bb7b40d None admin] RBAC: Adding query filter params (name=compute) {{(pid=86827) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:35:48.008923 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4cf3ce-9b7e-4099-89e5-40e16bb7b40d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:48.010432 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4cf3ce-9b7e-4099-89e5-40e16bb7b40d None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:48.016302 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 510/997] 10.222.0.36 () {56 vars in 1112 bytes} [Wed Jul 27 10:35:47 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 10:35:48.021569 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4cf3ce-9b7e-4099-89e5-40e16bb7b40d None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:48.027631 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2cc869-b87d-4dcd-b96a-cd3953669763 None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:48.035629 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:48.066630 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d75936-5f13-4407-ac22-5ee3abd19538 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:48.067465 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d75936-5f13-4407-ac22-5ee3abd19538 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:48.067747 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d75936-5f13-4407-ac22-5ee3abd19538 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:48.068120 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d75936-5f13-4407-ac22-5ee3abd19538 None admin] PATH_INFO: `/v3/services` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:48.068880 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d75936-5f13-4407-ac22-5ee3abd19538 None admin] RBAC: Adding query filter params (type=image) {{(pid=86828) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:35:48.069271 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d75936-5f13-4407-ac22-5ee3abd19538 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:48.070739 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d75936-5f13-4407-ac22-5ee3abd19538 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:48.075653 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 488/998] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 10:35:47 2022] GET /identity/v3/services?type=image => generated 122 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 10:35:48.106093 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:48.145311 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4cf3ce-9b7e-4099-89e5-40e16bb7b40d None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:48.184104 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e33c9b4-9e28-4785-bc92-b8183b8adefa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:48.184872 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e33c9b4-9e28-4785-bc92-b8183b8adefa None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:48.185225 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e33c9b4-9e28-4785-bc92-b8183b8adefa None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:48.185575 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e33c9b4-9e28-4785-bc92-b8183b8adefa None admin] PATH_INFO: `/v3/services` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:48.186346 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e33c9b4-9e28-4785-bc92-b8183b8adefa None admin] RBAC: Adding query filter params (type=compute) {{(pid=86827) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:35:48.186773 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e33c9b4-9e28-4785-bc92-b8183b8adefa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:48.188261 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e33c9b4-9e28-4785-bc92-b8183b8adefa None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:48.194427 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 511/999] 10.222.0.36 () {56 vars in 1112 bytes} [Wed Jul 27 10:35:48 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 10:35:48.200167 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d75936-5f13-4407-ac22-5ee3abd19538 None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:48.307979 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d75936-5f13-4407-ac22-5ee3abd19538 None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:48.342241 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad83de4-5e08-4175-94c9-64ff49461452 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:48.342897 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad83de4-5e08-4175-94c9-64ff49461452 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:48.343183 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad83de4-5e08-4175-94c9-64ff49461452 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:48.343461 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad83de4-5e08-4175-94c9-64ff49461452 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:48.344219 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad83de4-5e08-4175-94c9-64ff49461452 None admin] RBAC: Adding query filter params (interface=public, service_id=13eac701f2ba404d943533cfdf8b0f15, region_id=RegionOne) {{(pid=86828) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:35:48.344533 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad83de4-5e08-4175-94c9-64ff49461452 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:48.345963 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad83de4-5e08-4175-94c9-64ff49461452 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:48.355784 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 489/1000] 10.222.0.36 () {56 vars in 1251 bytes} [Wed Jul 27 10:35:48 2022] GET /identity/v3/endpoints?service_id=13eac701f2ba404d943533cfdf8b0f15&interface=public®ion_id=RegionOne => generated 194 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:35:48.356975 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e33c9b4-9e28-4785-bc92-b8183b8adefa None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:48.456971 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e33c9b4-9e28-4785-bc92-b8183b8adefa None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:48.486931 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:48.493574 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b68ef5-e05a-4e94-8d1f-798a256a983e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:48.494201 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b68ef5-e05a-4e94-8d1f-798a256a983e None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:48.494486 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b68ef5-e05a-4e94-8d1f-798a256a983e None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:48.494794 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b68ef5-e05a-4e94-8d1f-798a256a983e None admin] PATH_INFO: `/v3/services` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:48.495456 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b68ef5-e05a-4e94-8d1f-798a256a983e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:48.496822 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b68ef5-e05a-4e94-8d1f-798a256a983e None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:48.510631 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 512/1001] 10.222.0.36 () {56 vars in 1087 bytes} [Wed Jul 27 10:35:48 2022] GET /identity/v3/services => generated 1727 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 10:35:49.516965 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcf3990-b4e8-4687-8493-610f0fbde318 None None] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:49.517512 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcf3990-b4e8-4687-8493-610f0fbde318 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:49.517512 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcf3990-b4e8-4687-8493-610f0fbde318 None None] PATH_INFO: `/` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:49.518936 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 490/1002] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 10:35:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 10:35:49.524594 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837b07db-6350-44c1-8357-a87ddad50f49 None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:49.524996 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837b07db-6350-44c1-8357-a87ddad50f49 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:49.525159 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837b07db-6350-44c1-8357-a87ddad50f49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:49.543169 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-837b07db-6350-44c1-8357-a87ddad50f49 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:35:49.544355 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-837b07db-6350-44c1-8357-a87ddad50f49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:35:49.614247 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 513/1003] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 10:35:49 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 10:35:49.636943 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcf3990-b4e8-4687-8493-610f0fbde318 None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:49.672405 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:49.739508 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcf3990-b4e8-4687-8493-610f0fbde318 None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:49.781320 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c054b0-8b6b-4405-84ea-2e4ebc4dc952 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:49.781893 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c054b0-8b6b-4405-84ea-2e4ebc4dc952 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:49.782176 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c054b0-8b6b-4405-84ea-2e4ebc4dc952 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:49.782519 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c054b0-8b6b-4405-84ea-2e4ebc4dc952 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:49.783219 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c054b0-8b6b-4405-84ea-2e4ebc4dc952 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:49.784593 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c054b0-8b6b-4405-84ea-2e4ebc4dc952 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:49.793819 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application [None req-69c054b0-8b6b-4405-84ea-2e4ebc4dc952 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 27 10:35:49.793819 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:35:49.793819 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 27 10:35:49.793819 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 27 10:35:49.793819 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 27 10:35:49.793819 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 27 10:35:49.793819 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 27 10:35:49.793819 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 10:35:49.793819 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 27 10:35:49.793819 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Jul 27 10:35:49.793819 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 10:35:49.793819 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Jul 27 10:35:49.793819 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:35:49.793819 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:35:49.793819 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:35:49.793819 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:35:49.793819 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:35:49.795063 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:35:49.795063 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:35:49.795063 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:35:49.795063 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:35:49.795063 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:35:49.795063 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:35:49.795063 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 27 10:35:49.795063 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 27 10:35:49.795063 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 27 10:35:49.795063 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 27 10:35:49.795063 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:35:49.795063 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:35:49.795063 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:35:49.795063 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:35:49.795063 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:35:49.795063 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:35:49.795925 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:35:49.795925 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application with Lock( Jul 27 10:35:49.795925 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:35:49.795925 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:35:49.795925 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:35:49.795925 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:35:49.795925 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:35:49.795925 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:35:49.795925 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:35:49.795925 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:35:49.795925 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 27 10:35:49.795925 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 10:35:49.795925 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 27 10:35:49.795925 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Jul 27 10:35:49.797130 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 491/1004] 10.222.0.36 () {56 vars in 1126 bytes} [Wed Jul 27 10:35:49 2022] GET /identity/v3/services/object-store => generated 93 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 10:35:49.801394 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837b07db-6350-44c1-8357-a87ddad50f49 None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:49.899783 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837b07db-6350-44c1-8357-a87ddad50f49 None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:49.924945 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:49.934875 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13853ffd-9167-47eb-a211-185434edc1c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:49.935497 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13853ffd-9167-47eb-a211-185434edc1c4 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:49.938729 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13853ffd-9167-47eb-a211-185434edc1c4 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:49.938729 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13853ffd-9167-47eb-a211-185434edc1c4 None admin] PATH_INFO: `/v3/services` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:49.938729 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13853ffd-9167-47eb-a211-185434edc1c4 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=86827) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:35:49.938729 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13853ffd-9167-47eb-a211-185434edc1c4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:49.938729 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13853ffd-9167-47eb-a211-185434edc1c4 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:49.950816 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 514/1005] 10.222.0.36 () {56 vars in 1122 bytes} [Wed Jul 27 10:35:49 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 10:35:49.952078 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c054b0-8b6b-4405-84ea-2e4ebc4dc952 None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:50.051459 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:50.056699 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c054b0-8b6b-4405-84ea-2e4ebc4dc952 None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:50.090797 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967c196e-624a-4795-a9f4-1e202b42c407 None None] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:50.091092 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967c196e-624a-4795-a9f4-1e202b42c407 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:50.091377 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967c196e-624a-4795-a9f4-1e202b42c407 None None] PATH_INFO: `/` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:50.092723 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660a0f98-6f66-4206-9884-7edca10cf71d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:50.092863 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 515/1006] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 10:35:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 10:35:50.093648 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660a0f98-6f66-4206-9884-7edca10cf71d None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:50.094104 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660a0f98-6f66-4206-9884-7edca10cf71d None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:50.094644 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660a0f98-6f66-4206-9884-7edca10cf71d None admin] PATH_INFO: `/v3/services` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:50.095499 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660a0f98-6f66-4206-9884-7edca10cf71d None admin] RBAC: Adding query filter params (type=object-store) {{(pid=86828) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:35:50.096020 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660a0f98-6f66-4206-9884-7edca10cf71d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:50.097548 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ad8d4a-81d8-4535-9c7c-567c63920224 None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:50.097748 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660a0f98-6f66-4206-9884-7edca10cf71d None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:50.097865 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ad8d4a-81d8-4535-9c7c-567c63920224 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:50.098102 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ad8d4a-81d8-4535-9c7c-567c63920224 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:50.104417 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 492/1007] 10.222.0.36 () {56 vars in 1122 bytes} [Wed Jul 27 10:35:49 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:35:50.110122 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660a0f98-6f66-4206-9884-7edca10cf71d None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:50.118341 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-23ad8d4a-81d8-4535-9c7c-567c63920224 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:35:50.119600 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-23ad8d4a-81d8-4535-9c7c-567c63920224 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:35:50.193104 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 516/1008] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 10:35:50 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 27 10:35:50.221720 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ad8d4a-81d8-4535-9c7c-567c63920224 None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:50.247093 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660a0f98-6f66-4206-9884-7edca10cf71d None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:50.282493 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292d7abb-d4f3-413c-a74e-0aeed26161e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:50.283140 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292d7abb-d4f3-413c-a74e-0aeed26161e8 None admin] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:50.283420 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292d7abb-d4f3-413c-a74e-0aeed26161e8 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:50.283688 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292d7abb-d4f3-413c-a74e-0aeed26161e8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:50.284390 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292d7abb-d4f3-413c-a74e-0aeed26161e8 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:50.285741 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292d7abb-d4f3-413c-a74e-0aeed26161e8 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:50.303868 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 493/1009] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Jul 27 10:35:50 2022] POST /identity/v3/endpoints => generated 325 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 10:35:50.329746 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ad8d4a-81d8-4535-9c7c-567c63920224 None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:50.353180 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:50.378348 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbd75a1-6eaf-4fc0-9865-2271b499b05f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:50.379040 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbd75a1-6eaf-4fc0-9865-2271b499b05f None admin] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:50.379324 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbd75a1-6eaf-4fc0-9865-2271b499b05f None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:50.379600 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbd75a1-6eaf-4fc0-9865-2271b499b05f None admin] PATH_INFO: `/v3/services` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:50.380306 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbd75a1-6eaf-4fc0-9865-2271b499b05f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:50.381794 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbd75a1-6eaf-4fc0-9865-2271b499b05f None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:50.399771 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 517/1010] 10.222.0.36 () {60 vars in 1141 bytes} [Wed Jul 27 10:35:50 2022] POST /identity/v3/services => generated 242 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 10:35:50.523483 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8935e8-9ff3-4be5-bd73-c57ca7b62c44 None None] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:50.524049 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8935e8-9ff3-4be5-bd73-c57ca7b62c44 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:50.524049 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8935e8-9ff3-4be5-bd73-c57ca7b62c44 None None] PATH_INFO: `/` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:50.525647 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 494/1011] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 10:35:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 10:35:50.532129 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2436c4d7-0bbf-44d9-bef2-ae5f6b0915ff None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:50.532631 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2436c4d7-0bbf-44d9-bef2-ae5f6b0915ff None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:50.532740 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2436c4d7-0bbf-44d9-bef2-ae5f6b0915ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:50.553190 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-2436c4d7-0bbf-44d9-bef2-ae5f6b0915ff None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:35:50.554481 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-2436c4d7-0bbf-44d9-bef2-ae5f6b0915ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:35:50.634476 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 518/1012] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 10:35:50 2022] POST /identity/v3/auth/tokens => generated 1803 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 10:35:50.657650 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8935e8-9ff3-4be5-bd73-c57ca7b62c44 None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:50.684143 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:50.766204 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8935e8-9ff3-4be5-bd73-c57ca7b62c44 None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:50.802032 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d536a17a-353e-4483-b200-668d1eec94f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:50.802804 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d536a17a-353e-4483-b200-668d1eec94f0 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:50.803089 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d536a17a-353e-4483-b200-668d1eec94f0 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:50.803366 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d536a17a-353e-4483-b200-668d1eec94f0 None admin] PATH_INFO: `/v3/services/compute` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:50.803995 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d536a17a-353e-4483-b200-668d1eec94f0 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:50.805428 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d536a17a-353e-4483-b200-668d1eec94f0 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:50.814442 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application [None req-d536a17a-353e-4483-b200-668d1eec94f0 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 27 10:35:50.814442 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:35:50.814442 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 27 10:35:50.814442 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 27 10:35:50.814442 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 27 10:35:50.814442 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 27 10:35:50.814442 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 27 10:35:50.814442 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 10:35:50.814442 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 27 10:35:50.814442 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Jul 27 10:35:50.814442 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 10:35:50.814442 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Jul 27 10:35:50.814442 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:35:50.814442 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:35:50.814442 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:35:50.814442 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:35:50.814442 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:35:50.815568 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:35:50.815568 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:35:50.815568 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:35:50.815568 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:35:50.815568 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:35:50.815568 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:35:50.815568 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 27 10:35:50.815568 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 27 10:35:50.815568 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 27 10:35:50.815568 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 27 10:35:50.815568 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:35:50.815568 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:35:50.815568 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:35:50.815568 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:35:50.815568 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:35:50.815568 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:35:50.816407 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:35:50.816407 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application with Lock( Jul 27 10:35:50.816407 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:35:50.816407 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:35:50.816407 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:35:50.816407 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:35:50.816407 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:35:50.816407 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:35:50.816407 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:35:50.816407 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:35:50.816407 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 27 10:35:50.816407 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 10:35:50.816407 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 27 10:35:50.816407 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Jul 27 10:35:50.816407 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 495/1013] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Jul 27 10:35:50 2022] GET /identity/v3/services/compute => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 10:35:50.821682 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2436c4d7-0bbf-44d9-bef2-ae5f6b0915ff None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:50.924066 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2436c4d7-0bbf-44d9-bef2-ae5f6b0915ff None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:50.925628 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:50.971576 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d11e78-2d48-4afb-8ecf-e82c3476bbb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:50.972206 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d11e78-2d48-4afb-8ecf-e82c3476bbb5 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:50.972489 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d11e78-2d48-4afb-8ecf-e82c3476bbb5 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:50.972765 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d11e78-2d48-4afb-8ecf-e82c3476bbb5 None admin] PATH_INFO: `/v3/services` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:50.973445 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d11e78-2d48-4afb-8ecf-e82c3476bbb5 None admin] RBAC: Adding query filter params (name=compute) {{(pid=86827) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:35:50.973753 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d11e78-2d48-4afb-8ecf-e82c3476bbb5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:50.975166 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d11e78-2d48-4afb-8ecf-e82c3476bbb5 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:50.984925 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 519/1014] 10.222.0.36 () {56 vars in 1112 bytes} [Wed Jul 27 10:35:50 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 10:35:50.986582 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d536a17a-353e-4483-b200-668d1eec94f0 None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:51.076077 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:51.091826 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d536a17a-353e-4483-b200-668d1eec94f0 None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:51.127515 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4154fcb1-68bf-48cc-b6b4-2a6784433e05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:51.128142 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4154fcb1-68bf-48cc-b6b4-2a6784433e05 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:51.128427 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4154fcb1-68bf-48cc-b6b4-2a6784433e05 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:51.128703 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4154fcb1-68bf-48cc-b6b4-2a6784433e05 None admin] PATH_INFO: `/v3/services` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:51.129385 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4154fcb1-68bf-48cc-b6b4-2a6784433e05 None admin] RBAC: Adding query filter params (type=compute) {{(pid=86828) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:35:51.129691 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4154fcb1-68bf-48cc-b6b4-2a6784433e05 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:51.131439 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4154fcb1-68bf-48cc-b6b4-2a6784433e05 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:51.141320 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 496/1015] 10.222.0.36 () {56 vars in 1112 bytes} [Wed Jul 27 10:35:50 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:35:51.142851 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d11e78-2d48-4afb-8ecf-e82c3476bbb5 None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:51.246139 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d11e78-2d48-4afb-8ecf-e82c3476bbb5 None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:51.279952 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d33b390-6a3f-43d6-ad4a-8b1ab6b82306 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:51.280570 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d33b390-6a3f-43d6-ad4a-8b1ab6b82306 None admin] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:51.280854 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d33b390-6a3f-43d6-ad4a-8b1ab6b82306 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:51.281127 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d33b390-6a3f-43d6-ad4a-8b1ab6b82306 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:51.281828 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d33b390-6a3f-43d6-ad4a-8b1ab6b82306 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:51.283229 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d33b390-6a3f-43d6-ad4a-8b1ab6b82306 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:51.310636 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 520/1016] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Jul 27 10:35:51 2022] POST /identity/v3/endpoints => generated 334 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 10:35:52.341615 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08199ff9-d847-4971-b51f-3cfacba677d5 None None] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:52.342177 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08199ff9-d847-4971-b51f-3cfacba677d5 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:52.342177 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08199ff9-d847-4971-b51f-3cfacba677d5 None None] PATH_INFO: `/` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:52.343559 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 497/1017] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 10:35:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 10:35:52.348556 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa899a4-467d-4ea0-8ba0-676c46dae076 None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:52.348967 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa899a4-467d-4ea0-8ba0-676c46dae076 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:52.349134 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa899a4-467d-4ea0-8ba0-676c46dae076 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:52.367384 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-0aa899a4-467d-4ea0-8ba0-676c46dae076 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:35:52.368573 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-0aa899a4-467d-4ea0-8ba0-676c46dae076 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:35:52.386556 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:52.446165 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 521/1018] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 10:35:52 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 10:35:52.457804 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdb083c-6b0f-443b-8286-af6939b0d539 None None] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:52.458097 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdb083c-6b0f-443b-8286-af6939b0d539 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:52.458379 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdb083c-6b0f-443b-8286-af6939b0d539 None None] PATH_INFO: `/` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:52.460454 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 498/1019] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 10:35:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 10:35:52.465437 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb83f87-2b12-49bc-a53a-d296c437bf97 None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:52.465732 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb83f87-2b12-49bc-a53a-d296c437bf97 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:52.466012 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb83f87-2b12-49bc-a53a-d296c437bf97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:52.468175 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdb083c-6b0f-443b-8286-af6939b0d539 None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:52.488075 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-2cb83f87-2b12-49bc-a53a-d296c437bf97 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:35:52.489283 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-2cb83f87-2b12-49bc-a53a-d296c437bf97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:35:52.565765 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 522/1020] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 10:35:52 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 10:35:52.583128 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdb083c-6b0f-443b-8286-af6939b0d539 None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:52.588814 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb83f87-2b12-49bc-a53a-d296c437bf97 None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:52.620548 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a8ab8e-4702-4379-91e3-e253c4dff2fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:52.621116 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a8ab8e-4702-4379-91e3-e253c4dff2fe None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:52.621401 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a8ab8e-4702-4379-91e3-e253c4dff2fe None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:52.621679 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a8ab8e-4702-4379-91e3-e253c4dff2fe None admin] PATH_INFO: `/v3/domains/default` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:52.622883 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a8ab8e-4702-4379-91e3-e253c4dff2fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:52.624314 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a8ab8e-4702-4379-91e3-e253c4dff2fe None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:52.626034 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 499/1021] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 10:35:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 10:35:52.632786 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a8ab8e-4702-4379-91e3-e253c4dff2fe None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:52.654364 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:52.681243 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:52.707071 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb83f87-2b12-49bc-a53a-d296c437bf97 None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:52.742916 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96aa59a-a64c-466d-a122-b3969e8b69a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:52.743314 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a8ab8e-4702-4379-91e3-e253c4dff2fe None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:52.743564 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96aa59a-a64c-466d-a122-b3969e8b69a0 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:52.743851 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96aa59a-a64c-466d-a122-b3969e8b69a0 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:52.744126 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96aa59a-a64c-466d-a122-b3969e8b69a0 None admin] PATH_INFO: `/v3/services/image` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:52.744758 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96aa59a-a64c-466d-a122-b3969e8b69a0 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:52.746177 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96aa59a-a64c-466d-a122-b3969e8b69a0 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:52.755036 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application [None req-c96aa59a-a64c-466d-a122-b3969e8b69a0 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 27 10:35:52.755036 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:35:52.755036 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 27 10:35:52.755036 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 27 10:35:52.755036 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 27 10:35:52.755036 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 27 10:35:52.755036 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 27 10:35:52.755036 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 10:35:52.755036 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 27 10:35:52.755036 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Jul 27 10:35:52.755036 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 10:35:52.755036 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Jul 27 10:35:52.755036 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:35:52.755036 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:35:52.755036 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:35:52.755036 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:35:52.755036 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:35:52.756321 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:35:52.756321 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:35:52.756321 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:35:52.756321 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:35:52.756321 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:35:52.756321 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:35:52.756321 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 27 10:35:52.756321 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 27 10:35:52.756321 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 27 10:35:52.756321 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 27 10:35:52.756321 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:35:52.756321 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:35:52.756321 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:35:52.756321 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:35:52.756321 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:35:52.756321 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:35:52.757494 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:35:52.757494 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application with Lock( Jul 27 10:35:52.757494 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:35:52.757494 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:35:52.757494 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:35:52.757494 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:35:52.757494 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:35:52.757494 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:35:52.757494 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:35:52.757494 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:35:52.757494 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 27 10:35:52.757494 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 10:35:52.757494 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 27 10:35:52.757494 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Jul 27 10:35:52.757494 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 523/1022] 10.222.0.36 () {56 vars in 1105 bytes} [Wed Jul 27 10:35:52 2022] GET /identity/v3/services/image => generated 86 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 10:35:52.762443 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96aa59a-a64c-466d-a122-b3969e8b69a0 None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:52.787141 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4e1764-baf9-474d-9e52-b499b9a742c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:52.788382 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4e1764-baf9-474d-9e52-b499b9a742c1 None admin] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:52.788727 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4e1764-baf9-474d-9e52-b499b9a742c1 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:52.789165 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4e1764-baf9-474d-9e52-b499b9a742c1 None admin] PATH_INFO: `/v3/projects` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:52.789912 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4e1764-baf9-474d-9e52-b499b9a742c1 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:52.791373 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4e1764-baf9-474d-9e52-b499b9a742c1 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:52.819299 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 500/1023] 10.222.0.36 () {60 vars in 1141 bytes} [Wed Jul 27 10:35:52 2022] POST /identity/v3/projects => generated 311 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 10:35:52.873962 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96aa59a-a64c-466d-a122-b3969e8b69a0 None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:52.911529 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e6ade7-a88a-42b4-9601-623c087664d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:52.912783 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e6ade7-a88a-42b4-9601-623c087664d7 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:52.913090 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e6ade7-a88a-42b4-9601-623c087664d7 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:52.913363 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e6ade7-a88a-42b4-9601-623c087664d7 None admin] PATH_INFO: `/v3/services` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:52.914061 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e6ade7-a88a-42b4-9601-623c087664d7 None admin] RBAC: Adding query filter params (name=image) {{(pid=86827) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:35:52.914367 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e6ade7-a88a-42b4-9601-623c087664d7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:52.915834 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e6ade7-a88a-42b4-9601-623c087664d7 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:52.921830 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 524/1024] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 10:35:52 2022] GET /identity/v3/services?name=image => generated 122 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:35:52.927470 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4e1764-baf9-474d-9e52-b499b9a742c1 None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:53.009413 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:53.044943 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4e1764-baf9-474d-9e52-b499b9a742c1 None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:53.078916 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16880790-909c-4724-b36d-7cfea50d8711 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:53.079552 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16880790-909c-4724-b36d-7cfea50d8711 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:53.079838 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16880790-909c-4724-b36d-7cfea50d8711 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:53.080115 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16880790-909c-4724-b36d-7cfea50d8711 None admin] PATH_INFO: `/v3/services` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:53.080796 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16880790-909c-4724-b36d-7cfea50d8711 None admin] RBAC: Adding query filter params (type=image) {{(pid=86828) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:35:53.081114 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16880790-909c-4724-b36d-7cfea50d8711 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:53.082486 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16880790-909c-4724-b36d-7cfea50d8711 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:53.093994 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 501/1025] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 10:35:52 2022] GET /identity/v3/services?type=image => generated 350 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:35:53.095695 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e6ade7-a88a-42b4-9601-623c087664d7 None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:53.106160 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:53.201734 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e6ade7-a88a-42b4-9601-623c087664d7 None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:53.236066 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5600bc03-fbcc-45e2-8b29-e8fc8b348a25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:53.236688 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5600bc03-fbcc-45e2-8b29-e8fc8b348a25 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:53.236968 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5600bc03-fbcc-45e2-8b29-e8fc8b348a25 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:53.237242 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5600bc03-fbcc-45e2-8b29-e8fc8b348a25 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:53.237996 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5600bc03-fbcc-45e2-8b29-e8fc8b348a25 None admin] RBAC: Adding query filter params (interface=public, service_id=fa76cdbcfdf740999565f93f400f6879, region_id=RegionOne) {{(pid=86827) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:35:53.238305 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5600bc03-fbcc-45e2-8b29-e8fc8b348a25 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:53.239699 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5600bc03-fbcc-45e2-8b29-e8fc8b348a25 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:53.249312 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 525/1026] 10.222.0.36 () {56 vars in 1251 bytes} [Wed Jul 27 10:35:53 2022] GET /identity/v3/endpoints?service_id=fa76cdbcfdf740999565f93f400f6879&interface=public®ion_id=RegionOne => generated 194 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:35:53.250581 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16880790-909c-4724-b36d-7cfea50d8711 None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:53.352198 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16880790-909c-4724-b36d-7cfea50d8711 None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:53.385939 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9966c65a-0182-48c1-8383-a68e28b82374 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:53.386565 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9966c65a-0182-48c1-8383-a68e28b82374 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:53.386874 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9966c65a-0182-48c1-8383-a68e28b82374 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:53.387148 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9966c65a-0182-48c1-8383-a68e28b82374 None admin] PATH_INFO: `/v3/services` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:53.387799 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9966c65a-0182-48c1-8383-a68e28b82374 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:53.389164 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9966c65a-0182-48c1-8383-a68e28b82374 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:53.390221 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:53.398413 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 502/1027] 10.222.0.36 () {56 vars in 1087 bytes} [Wed Jul 27 10:35:53 2022] GET /identity/v3/services => generated 1957 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 10:35:53.435160 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ca0b5c-37db-44ac-bec8-8bf362ab255d None None] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:53.435680 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ca0b5c-37db-44ac-bec8-8bf362ab255d None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:53.435680 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ca0b5c-37db-44ac-bec8-8bf362ab255d None None] PATH_INFO: `/` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:53.437089 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 526/1028] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 10:35:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 10:35:53.443445 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da71fe8c-1974-498c-a886-dbc4955ff00e None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:53.443748 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da71fe8c-1974-498c-a886-dbc4955ff00e None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:53.443995 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da71fe8c-1974-498c-a886-dbc4955ff00e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:53.468263 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-da71fe8c-1974-498c-a886-dbc4955ff00e None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:35:53.469381 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-da71fe8c-1974-498c-a886-dbc4955ff00e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:35:53.553828 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 503/1029] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 10:35:53 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 10:35:53.576976 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ca0b5c-37db-44ac-bec8-8bf362ab255d None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:53.653911 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:53.688930 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ca0b5c-37db-44ac-bec8-8bf362ab255d None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:53.722941 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56624512-d4f1-449a-ba5e-17311df06df6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:53.723508 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56624512-d4f1-449a-ba5e-17311df06df6 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:53.723790 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56624512-d4f1-449a-ba5e-17311df06df6 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:53.724067 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56624512-d4f1-449a-ba5e-17311df06df6 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:53.735326 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56624512-d4f1-449a-ba5e-17311df06df6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 10:35:53.740811 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application [None req-56624512-d4f1-449a-ba5e-17311df06df6 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 27 10:35:53.740811 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:35:53.740811 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:35:53.740811 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:35:53.740811 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:35:53.740811 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:35:53.740811 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:35:53.740811 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:35:53.740811 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:35:53.740811 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:35:53.740811 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:35:53.740811 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:35:53.740811 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 10:35:53.740811 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 10:35:53.740811 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 10:35:53.740811 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 10:35:53.741724 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 10:35:53.741724 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 10:35:53.741724 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 10:35:53.741724 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 10:35:53.741724 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:35:53.741724 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:35:53.741724 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:35:53.741724 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:35:53.741724 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:35:53.741724 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:35:53.741724 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:35:53.741724 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application with Lock( Jul 27 10:35:53.741724 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:35:53.741724 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:35:53.741724 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:35:53.741724 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:35:53.742567 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:35:53.742567 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:35:53.742567 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:35:53.742567 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:35:53.742567 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 10:35:53.742567 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 10:35:53.742567 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 10:35:53.742567 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 10:35:53.742567 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 10:35:53.742567 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 10:35:53.742567 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 27 10:35:53.742567 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Jul 27 10:35:53.742567 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 527/1030] 10.222.0.36 () {56 vars in 1120 bytes} [Wed Jul 27 10:35:53 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 10:35:53.750138 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da71fe8c-1974-498c-a886-dbc4955ff00e None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:53.853744 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da71fe8c-1974-498c-a886-dbc4955ff00e None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:53.881004 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:53.890263 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e4aba2-070c-4f77-a09a-07558fc2db70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:53.890934 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e4aba2-070c-4f77-a09a-07558fc2db70 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:53.891221 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e4aba2-070c-4f77-a09a-07558fc2db70 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:53.891496 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e4aba2-070c-4f77-a09a-07558fc2db70 None admin] PATH_INFO: `/v3/roles` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:53.892225 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e4aba2-070c-4f77-a09a-07558fc2db70 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=86828) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:35:53.892532 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e4aba2-070c-4f77-a09a-07558fc2db70 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:53.893900 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e4aba2-070c-4f77-a09a-07558fc2db70 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:53.906543 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 504/1031] 10.222.0.36 () {56 vars in 1115 bytes} [Wed Jul 27 10:35:53 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 10:35:53.908321 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56624512-d4f1-449a-ba5e-17311df06df6 None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:54.020981 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56624512-d4f1-449a-ba5e-17311df06df6 None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:54.050077 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:54.058419 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c97ceb-4084-46ad-8cd9-dab9e68950c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:54.059097 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c97ceb-4084-46ad-8cd9-dab9e68950c6 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:54.059377 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c97ceb-4084-46ad-8cd9-dab9e68950c6 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:54.059649 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c97ceb-4084-46ad-8cd9-dab9e68950c6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:54.070516 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c97ceb-4084-46ad-8cd9-dab9e68950c6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:54.071983 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c97ceb-4084-46ad-8cd9-dab9e68950c6 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:54.086246 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application [None req-53c97ceb-4084-46ad-8cd9-dab9e68950c6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 10:35:54.086246 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:35:54.086246 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:35:54.086246 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:35:54.086246 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:35:54.086246 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:35:54.086246 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:35:54.086246 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:35:54.086246 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:35:54.086246 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:35:54.086246 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:35:54.086246 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:35:54.086246 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 10:35:54.086246 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 10:35:54.086246 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 10:35:54.087166 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 10:35:54.087166 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:35:54.087166 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:35:54.087166 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:35:54.087166 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:35:54.087166 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:35:54.087166 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:35:54.087166 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:35:54.087166 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application with Lock( Jul 27 10:35:54.087166 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:35:54.087166 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:35:54.087166 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:35:54.087166 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:35:54.087166 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:35:54.087166 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:35:54.087166 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:35:54.087166 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:35:54.088053 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 10:35:54.088053 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 10:35:54.088053 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 10:35:54.088053 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Jul 27 10:35:54.090912 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 528/1032] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 10:35:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 10:35:54.093178 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e4aba2-070c-4f77-a09a-07558fc2db70 None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:54.196434 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e4aba2-070c-4f77-a09a-07558fc2db70 None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:54.232776 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fb3c55-0b4e-4d62-9151-5707fae76d26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:54.233411 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fb3c55-0b4e-4d62-9151-5707fae76d26 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:54.233697 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fb3c55-0b4e-4d62-9151-5707fae76d26 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:54.233973 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fb3c55-0b4e-4d62-9151-5707fae76d26 None admin] PATH_INFO: `/v3/domains` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:54.234705 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84fb3c55-0b4e-4d62-9151-5707fae76d26 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86828) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:35:54.235016 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84fb3c55-0b4e-4d62-9151-5707fae76d26 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:54.236384 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84fb3c55-0b4e-4d62-9151-5707fae76d26 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:54.256509 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 505/1033] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 10:35:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 10:35:54.258328 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c97ceb-4084-46ad-8cd9-dab9e68950c6 None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:54.359251 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c97ceb-4084-46ad-8cd9-dab9e68950c6 None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:54.393327 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5199a2f3-ff99-42b5-b226-a2e36a785862 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:54.393969 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5199a2f3-ff99-42b5-b226-a2e36a785862 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:54.394255 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5199a2f3-ff99-42b5-b226-a2e36a785862 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:54.394532 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5199a2f3-ff99-42b5-b226-a2e36a785862 None admin] PATH_INFO: `/v3/users/nova` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:54.402992 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5199a2f3-ff99-42b5-b226-a2e36a785862 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:54.404548 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5199a2f3-ff99-42b5-b226-a2e36a785862 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:54.416007 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application [None req-5199a2f3-ff99-42b5-b226-a2e36a785862 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 27 10:35:54.416007 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:35:54.416007 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:35:54.416007 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:35:54.416007 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:35:54.416007 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:35:54.416007 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:35:54.416007 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:35:54.416007 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:35:54.416007 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:35:54.416007 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:35:54.416007 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:35:54.416007 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 10:35:54.416007 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 10:35:54.416007 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 10:35:54.416007 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 10:35:54.416961 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:35:54.416961 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:35:54.416961 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 10:35:54.416961 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 10:35:54.416961 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 10:35:54.416961 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 10:35:54.416961 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:35:54.416961 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:35:54.416961 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:35:54.416961 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:35:54.416961 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:35:54.416961 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application with Lock( Jul 27 10:35:54.416961 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:35:54.416961 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:35:54.416961 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:35:54.416961 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:35:54.416961 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:35:54.416961 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:35:54.422896 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:35:54.422896 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:35:54.422896 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 10:35:54.422896 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 10:35:54.422896 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 10:35:54.422896 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 10:35:54.422896 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 10:35:54.422896 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 10:35:54.422896 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 27 10:35:54.422896 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Jul 27 10:35:54.422896 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 529/1034] 10.222.0.36 () {56 vars in 1093 bytes} [Wed Jul 27 10:35:54 2022] GET /identity/v3/users/nova => generated 82 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 10:35:54.425158 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fb3c55-0b4e-4d62-9151-5707fae76d26 None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:54.477947 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:54.530947 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fb3c55-0b4e-4d62-9151-5707fae76d26 None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:54.565652 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b2d1c5-5e66-4cec-b57e-178bf8fd7026 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:54.566274 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b2d1c5-5e66-4cec-b57e-178bf8fd7026 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:54.566552 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b2d1c5-5e66-4cec-b57e-178bf8fd7026 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:54.566855 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b2d1c5-5e66-4cec-b57e-178bf8fd7026 None admin] PATH_INFO: `/v3/users` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:54.567654 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b2d1c5-5e66-4cec-b57e-178bf8fd7026 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86828) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:35:54.568019 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b2d1c5-5e66-4cec-b57e-178bf8fd7026 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:54.569974 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b2d1c5-5e66-4cec-b57e-178bf8fd7026 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:54.594261 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 506/1035] 10.222.0.36 () {56 vars in 1133 bytes} [Wed Jul 27 10:35:54 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:35:54.596249 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5199a2f3-ff99-42b5-b226-a2e36a785862 None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:54.645129 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:54.705017 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5199a2f3-ff99-42b5-b226-a2e36a785862 None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:54.738728 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff78cdc-e5b3-4dd2-8a0f-837d995f87ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:54.739348 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff78cdc-e5b3-4dd2-8a0f-837d995f87ed None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:54.739634 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff78cdc-e5b3-4dd2-8a0f-837d995f87ed None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:54.740477 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff78cdc-e5b3-4dd2-8a0f-837d995f87ed None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:54.811377 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff78cdc-e5b3-4dd2-8a0f-837d995f87ed None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:54.812836 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff78cdc-e5b3-4dd2-8a0f-837d995f87ed None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:54.855508 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:54.894682 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e386e8ef-016e-461d-9160-a8568111cfd2 None None] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:54.895025 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e386e8ef-016e-461d-9160-a8568111cfd2 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:54.895305 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e386e8ef-016e-461d-9160-a8568111cfd2 None None] PATH_INFO: `/` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:54.897339 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 507/1036] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 10:35:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 10:35:54.902372 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20da4c1-02ec-4c96-8681-7e8ce0fb1c85 None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:54.902686 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20da4c1-02ec-4c96-8681-7e8ce0fb1c85 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:54.902970 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20da4c1-02ec-4c96-8681-7e8ce0fb1c85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:54.920466 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-c20da4c1-02ec-4c96-8681-7e8ce0fb1c85 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:35:54.921668 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-c20da4c1-02ec-4c96-8681-7e8ce0fb1c85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:35:54.921837 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 530/1037] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jul 27 10:35:54 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 27 10:35:54.927023 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff78cdc-e5b3-4dd2-8a0f-837d995f87ed None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:54.996106 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 508/1038] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 10:35:54 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 27 10:35:55.020909 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20da4c1-02ec-4c96-8681-7e8ce0fb1c85 None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:55.037620 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff78cdc-e5b3-4dd2-8a0f-837d995f87ed None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:55.059569 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:55.077253 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1723a7-debf-49f6-83c7-96c9562feb5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:55.078260 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1723a7-debf-49f6-83c7-96c9562feb5f None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:55.078874 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1723a7-debf-49f6-83c7-96c9562feb5f None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:55.079158 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1723a7-debf-49f6-83c7-96c9562feb5f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:55.090884 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1723a7-debf-49f6-83c7-96c9562feb5f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:55.092397 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1723a7-debf-49f6-83c7-96c9562feb5f None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:55.094329 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:55.110829 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application [None req-fe1723a7-debf-49f6-83c7-96c9562feb5f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 10:35:55.110829 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:35:55.110829 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:35:55.110829 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:35:55.110829 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:35:55.110829 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:35:55.110829 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:35:55.110829 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:35:55.110829 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:35:55.110829 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:35:55.110829 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:35:55.110829 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:35:55.110829 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 10:35:55.110829 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 10:35:55.110829 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 10:35:55.111869 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 10:35:55.111869 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:35:55.111869 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:35:55.111869 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:35:55.111869 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:35:55.111869 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:35:55.111869 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:35:55.111869 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:35:55.111869 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application with Lock( Jul 27 10:35:55.111869 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:35:55.111869 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:35:55.111869 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:35:55.111869 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:35:55.111869 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:35:55.111869 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:35:55.111869 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:35:55.111869 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:35:55.112759 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 10:35:55.112759 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 10:35:55.112759 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 10:35:55.112759 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Jul 27 10:35:55.112759 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 531/1039] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 10:35:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 10:35:55.118328 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1723a7-debf-49f6-83c7-96c9562feb5f None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:55.124207 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20da4c1-02ec-4c96-8681-7e8ce0fb1c85 None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:55.160650 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031b5665-eaad-42b6-9469-a5871bfb9aba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:55.161216 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031b5665-eaad-42b6-9469-a5871bfb9aba None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:55.161496 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031b5665-eaad-42b6-9469-a5871bfb9aba None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:55.161799 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031b5665-eaad-42b6-9469-a5871bfb9aba None admin] PATH_INFO: `/v3/domains/default` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:55.162992 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031b5665-eaad-42b6-9469-a5871bfb9aba None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:55.164428 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031b5665-eaad-42b6-9469-a5871bfb9aba None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:55.166029 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 509/1040] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 10:35:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 10:35:55.171690 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031b5665-eaad-42b6-9469-a5871bfb9aba None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:55.236989 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1723a7-debf-49f6-83c7-96c9562feb5f None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:55.273983 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031b5665-eaad-42b6-9469-a5871bfb9aba None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:55.274926 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59104932-3158-4546-9c50-517919e4615f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:55.275556 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59104932-3158-4546-9c50-517919e4615f None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:55.275840 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59104932-3158-4546-9c50-517919e4615f None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:55.276110 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59104932-3158-4546-9c50-517919e4615f None admin] PATH_INFO: `/v3/domains` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:55.276816 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59104932-3158-4546-9c50-517919e4615f None admin] RBAC: Adding query filter params (name=Default) {{(pid=86827) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:35:55.277125 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59104932-3158-4546-9c50-517919e4615f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:55.278513 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59104932-3158-4546-9c50-517919e4615f None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:55.281985 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:55.296425 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 532/1041] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 10:35:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 10:35:55.297669 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59104932-3158-4546-9c50-517919e4615f None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:55.317866 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ec80bc-06de-4227-ac9d-866e45dcdb1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:55.319067 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ec80bc-06de-4227-ac9d-866e45dcdb1e None admin] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:55.319350 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ec80bc-06de-4227-ac9d-866e45dcdb1e None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:55.319624 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ec80bc-06de-4227-ac9d-866e45dcdb1e None admin] PATH_INFO: `/v3/users` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:55.320302 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ec80bc-06de-4227-ac9d-866e45dcdb1e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:55.321731 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ec80bc-06de-4227-ac9d-866e45dcdb1e None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:55.345222 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 510/1042] 10.222.0.36 () {60 vars in 1132 bytes} [Wed Jul 27 10:35:55 2022] POST /identity/v3/users => generated 285 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 10:35:55.361735 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:55.414790 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59104932-3158-4546-9c50-517919e4615f None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:55.451425 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1d74de-0ef7-4d70-b986-f9f50ab02e07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:55.452084 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1d74de-0ef7-4d70-b986-f9f50ab02e07 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:55.452367 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1d74de-0ef7-4d70-b986-f9f50ab02e07 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:55.452643 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1d74de-0ef7-4d70-b986-f9f50ab02e07 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:55.460381 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1d74de-0ef7-4d70-b986-f9f50ab02e07 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:55.461864 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1d74de-0ef7-4d70-b986-f9f50ab02e07 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:55.474685 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application [None req-2e1d74de-0ef7-4d70-b986-f9f50ab02e07 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 10:35:55.474685 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:35:55.474685 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:35:55.474685 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:35:55.474685 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:35:55.474685 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:35:55.474685 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:35:55.474685 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:35:55.474685 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:35:55.474685 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:35:55.474685 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:35:55.474685 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:35:55.474685 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 10:35:55.474685 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 10:35:55.474685 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 10:35:55.483565 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 10:35:55.483565 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:35:55.483565 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:35:55.483565 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:35:55.483565 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:35:55.483565 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:35:55.483565 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:35:55.483565 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:35:55.483565 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application with Lock( Jul 27 10:35:55.483565 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:35:55.483565 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:35:55.483565 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:35:55.483565 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:35:55.483565 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:35:55.483565 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:35:55.483565 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:35:55.483565 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:35:55.485376 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 10:35:55.485376 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 10:35:55.485376 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 10:35:55.485376 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 10:35:55.485376 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 10:35:55.485376 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 10:35:55.485376 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 10:35:55.485376 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Jul 27 10:35:55.485376 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 533/1043] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Jul 27 10:35:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 10:35:55.490572 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ec80bc-06de-4227-ac9d-866e45dcdb1e None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:55.516043 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252467f1-8804-4a8e-b34b-b5ddf60cfdf7 None None] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:55.516322 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252467f1-8804-4a8e-b34b-b5ddf60cfdf7 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:55.516597 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252467f1-8804-4a8e-b34b-b5ddf60cfdf7 None None] PATH_INFO: `/` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:55.518135 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 534/1044] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 10:35:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 10:35:55.522966 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9979eaa-57d2-4e35-b76b-9cd90d69a545 None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:55.523330 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9979eaa-57d2-4e35-b76b-9cd90d69a545 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:55.523612 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9979eaa-57d2-4e35-b76b-9cd90d69a545 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:55.542008 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-a9979eaa-57d2-4e35-b76b-9cd90d69a545 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:35:55.543658 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-a9979eaa-57d2-4e35-b76b-9cd90d69a545 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:35:55.620806 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ec80bc-06de-4227-ac9d-866e45dcdb1e None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:55.621686 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 535/1045] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 10:35:55 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 27 10:35:55.643670 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9979eaa-57d2-4e35-b76b-9cd90d69a545 None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:55.657255 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd856508-e60e-410b-89d1-2136a16ccbd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:55.657885 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd856508-e60e-410b-89d1-2136a16ccbd2 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:55.658167 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd856508-e60e-410b-89d1-2136a16ccbd2 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:55.658522 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd856508-e60e-410b-89d1-2136a16ccbd2 None admin] PATH_INFO: `/v3/projects` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:55.659224 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd856508-e60e-410b-89d1-2136a16ccbd2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86828) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:35:55.659531 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd856508-e60e-410b-89d1-2136a16ccbd2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:55.660963 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd856508-e60e-410b-89d1-2136a16ccbd2 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:55.677130 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 511/1046] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Jul 27 10:35:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:35:55.683896 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd856508-e60e-410b-89d1-2136a16ccbd2 None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:55.692733 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:55.713070 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:55.760438 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9979eaa-57d2-4e35-b76b-9cd90d69a545 None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:55.792422 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd856508-e60e-410b-89d1-2136a16ccbd2 None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:55.796108 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b08305-664d-4c6f-934a-8f318fcd31d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:55.796755 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b08305-664d-4c6f-934a-8f318fcd31d0 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:55.797036 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b08305-664d-4c6f-934a-8f318fcd31d0 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:55.797310 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b08305-664d-4c6f-934a-8f318fcd31d0 None admin] PATH_INFO: `/v3/services/image` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:55.798003 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b08305-664d-4c6f-934a-8f318fcd31d0 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:55.799433 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b08305-664d-4c6f-934a-8f318fcd31d0 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:55.809176 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application [None req-b6b08305-664d-4c6f-934a-8f318fcd31d0 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 27 10:35:55.809176 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:35:55.809176 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 27 10:35:55.809176 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 27 10:35:55.809176 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 27 10:35:55.809176 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 27 10:35:55.809176 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 27 10:35:55.809176 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 10:35:55.809176 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 27 10:35:55.809176 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Jul 27 10:35:55.809176 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 10:35:55.809176 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Jul 27 10:35:55.809176 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:35:55.809176 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:35:55.809176 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:35:55.809176 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:35:55.809176 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:35:55.810182 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:35:55.810182 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:35:55.810182 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:35:55.810182 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:35:55.810182 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:35:55.810182 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:35:55.810182 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 27 10:35:55.810182 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 27 10:35:55.810182 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 27 10:35:55.810182 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 27 10:35:55.810182 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:35:55.810182 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:35:55.810182 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:35:55.810182 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:35:55.810182 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:35:55.810182 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:35:55.811058 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:35:55.811058 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application with Lock( Jul 27 10:35:55.811058 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:35:55.811058 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:35:55.811058 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:35:55.811058 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:35:55.811058 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:35:55.811058 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:35:55.811058 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:35:55.811058 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:35:55.811058 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 27 10:35:55.811058 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 10:35:55.811058 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 27 10:35:55.811058 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Jul 27 10:35:55.811058 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 536/1047] 10.222.0.36 () {56 vars in 1105 bytes} [Wed Jul 27 10:35:55 2022] GET /identity/v3/services/image => generated 86 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 10:35:55.816819 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b08305-664d-4c6f-934a-8f318fcd31d0 None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:55.831128 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28567042-a1cb-410e-af44-eec8d4e3e70d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:55.831755 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28567042-a1cb-410e-af44-eec8d4e3e70d None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:55.832036 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28567042-a1cb-410e-af44-eec8d4e3e70d None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:55.832311 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28567042-a1cb-410e-af44-eec8d4e3e70d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:55.833108 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28567042-a1cb-410e-af44-eec8d4e3e70d None admin] RBAC: Adding query filter params (role.id=722097dd0463434b9c4ecaebe7ecf9ea, scope.project.id=5f296d8e3b8f4001909addc49d8dac01, user.id=e059d8b6c28c4fcba925d05d7c98e4ac) {{(pid=86828) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:35:55.833420 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28567042-a1cb-410e-af44-eec8d4e3e70d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:55.834906 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28567042-a1cb-410e-af44-eec8d4e3e70d None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:55.840519 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 512/1048] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Jul 27 10:35:55 2022] GET /identity/v3/role_assignments?user.id=e059d8b6c28c4fcba925d05d7c98e4ac&scope.project.id=5f296d8e3b8f4001909addc49d8dac01&role.id=722097dd0463434b9c4ecaebe7ecf9ea => generated 259 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:35:55.930267 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b08305-664d-4c6f-934a-8f318fcd31d0 None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:55.944155 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:55.975880 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61e0854-2f79-462f-8843-f4f29c002021 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:55.976612 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61e0854-2f79-462f-8843-f4f29c002021 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:55.976962 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61e0854-2f79-462f-8843-f4f29c002021 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:55.977307 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61e0854-2f79-462f-8843-f4f29c002021 None admin] PATH_INFO: `/v3/services` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:55.978071 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61e0854-2f79-462f-8843-f4f29c002021 None admin] RBAC: Adding query filter params (name=image) {{(pid=86827) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:35:55.978447 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61e0854-2f79-462f-8843-f4f29c002021 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:55.979930 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61e0854-2f79-462f-8843-f4f29c002021 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:55.985762 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 537/1049] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 10:35:55 2022] GET /identity/v3/services?name=image => generated 122 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 10:35:55.991957 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28567042-a1cb-410e-af44-eec8d4e3e70d None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:56.079208 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:56.095372 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28567042-a1cb-410e-af44-eec8d4e3e70d None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:56.130971 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2b3c46-2737-4935-bfb0-58d3a1037b63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:56.131695 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2b3c46-2737-4935-bfb0-58d3a1037b63 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:56.132052 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2b3c46-2737-4935-bfb0-58d3a1037b63 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:56.132401 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2b3c46-2737-4935-bfb0-58d3a1037b63 None admin] PATH_INFO: `/v3/services` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:56.133150 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2b3c46-2737-4935-bfb0-58d3a1037b63 None admin] RBAC: Adding query filter params (type=image) {{(pid=86828) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:35:56.133784 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2b3c46-2737-4935-bfb0-58d3a1037b63 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:56.135251 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2b3c46-2737-4935-bfb0-58d3a1037b63 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:56.140679 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 513/1050] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 10:35:55 2022] GET /identity/v3/services?type=image => generated 350 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:35:56.146179 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61e0854-2f79-462f-8843-f4f29c002021 None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:56.246087 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61e0854-2f79-462f-8843-f4f29c002021 None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:56.279181 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef239f3c-3023-4dbd-9c54-01521d9aa873 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:56.279801 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef239f3c-3023-4dbd-9c54-01521d9aa873 None admin] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:56.280083 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef239f3c-3023-4dbd-9c54-01521d9aa873 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:56.280357 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef239f3c-3023-4dbd-9c54-01521d9aa873 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:56.281077 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef239f3c-3023-4dbd-9c54-01521d9aa873 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:56.282438 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef239f3c-3023-4dbd-9c54-01521d9aa873 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:56.305343 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 538/1051] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Jul 27 10:35:56 2022] POST /identity/v3/endpoints => generated 327 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 10:35:57.391907 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274ccbbc-97c0-4be5-aec6-093d64c35570 None None] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:57.392447 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274ccbbc-97c0-4be5-aec6-093d64c35570 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:57.392447 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274ccbbc-97c0-4be5-aec6-093d64c35570 None None] PATH_INFO: `/` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:57.394640 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 514/1052] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 10:35:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 10:35:57.399000 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341d95b3-e8fc-49a6-8e51-448afd71b4dc None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:57.399447 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341d95b3-e8fc-49a6-8e51-448afd71b4dc None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:57.399616 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341d95b3-e8fc-49a6-8e51-448afd71b4dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:57.419157 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-341d95b3-e8fc-49a6-8e51-448afd71b4dc None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:35:57.420353 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-341d95b3-e8fc-49a6-8e51-448afd71b4dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:35:57.445535 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:57.498867 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 539/1053] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 10:35:57 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 10:35:57.521992 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274ccbbc-97c0-4be5-aec6-093d64c35570 None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:57.623806 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274ccbbc-97c0-4be5-aec6-093d64c35570 None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:57.656573 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2316e3-6909-495c-85c0-2ebe6d71afc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:57.657282 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2316e3-6909-495c-85c0-2ebe6d71afc1 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:57.657565 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2316e3-6909-495c-85c0-2ebe6d71afc1 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:57.657842 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2316e3-6909-495c-85c0-2ebe6d71afc1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:57.669476 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2316e3-6909-495c-85c0-2ebe6d71afc1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 10:35:57.674420 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application [None req-0f2316e3-6909-495c-85c0-2ebe6d71afc1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 10:35:57.674420 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:35:57.674420 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:35:57.674420 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:35:57.674420 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:35:57.674420 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:35:57.674420 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:35:57.674420 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:35:57.674420 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:35:57.674420 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:35:57.674420 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:35:57.674420 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:35:57.674420 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 10:35:57.674420 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 10:35:57.674420 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 10:35:57.674420 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 10:35:57.675369 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 10:35:57.675369 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 10:35:57.675369 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 10:35:57.675369 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 10:35:57.675369 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:35:57.675369 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:35:57.675369 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:35:57.675369 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:35:57.675369 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:35:57.675369 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:35:57.675369 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:35:57.675369 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application with Lock( Jul 27 10:35:57.675369 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:35:57.675369 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:35:57.675369 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:35:57.675369 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:35:57.676208 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:35:57.676208 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:35:57.676208 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:35:57.676208 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:35:57.676208 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 10:35:57.676208 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 10:35:57.676208 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 10:35:57.676208 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 10:35:57.676208 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 10:35:57.676208 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 10:35:57.676208 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 10:35:57.676208 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Jul 27 10:35:57.678908 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 515/1054] 10.222.0.36 () {56 vars in 1096 bytes} [Wed Jul 27 10:35:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 10:35:57.682087 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341d95b3-e8fc-49a6-8e51-448afd71b4dc None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:57.779538 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341d95b3-e8fc-49a6-8e51-448afd71b4dc None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:57.813123 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894066b5-055d-4b59-9323-86fdc314589f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:57.813890 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894066b5-055d-4b59-9323-86fdc314589f None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:57.814169 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894066b5-055d-4b59-9323-86fdc314589f None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:57.814440 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894066b5-055d-4b59-9323-86fdc314589f None admin] PATH_INFO: `/v3/roles` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:57.815249 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894066b5-055d-4b59-9323-86fdc314589f None admin] RBAC: Adding query filter params (name=admin) {{(pid=86827) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:35:57.815551 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894066b5-055d-4b59-9323-86fdc314589f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:57.816915 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894066b5-055d-4b59-9323-86fdc314589f None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:57.830266 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 540/1055] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jul 27 10:35:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:35:57.831676 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2316e3-6909-495c-85c0-2ebe6d71afc1 None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:57.845016 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:57.855722 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0980b99-44d0-4d18-997a-27373a80ca03 None None] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:57.856011 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0980b99-44d0-4d18-997a-27373a80ca03 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:57.856288 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0980b99-44d0-4d18-997a-27373a80ca03 None None] PATH_INFO: `/` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:57.857644 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 541/1056] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 10:35:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 10:35:57.862736 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dad4b7f-faec-42e4-a8bd-9440fdaa0559 None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:57.863026 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dad4b7f-faec-42e4-a8bd-9440fdaa0559 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:57.863305 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dad4b7f-faec-42e4-a8bd-9440fdaa0559 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:57.882380 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-9dad4b7f-faec-42e4-a8bd-9440fdaa0559 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:35:57.883867 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-9dad4b7f-faec-42e4-a8bd-9440fdaa0559 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:35:57.895317 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:57.940869 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2316e3-6909-495c-85c0-2ebe6d71afc1 None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:57.964772 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 542/1057] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 10:35:57 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 27 10:35:57.983131 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038cefc5-6157-4da0-a360-6cf413a73023 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:57.984108 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038cefc5-6157-4da0-a360-6cf413a73023 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:57.984629 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038cefc5-6157-4da0-a360-6cf413a73023 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:57.985133 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038cefc5-6157-4da0-a360-6cf413a73023 None admin] PATH_INFO: `/v3/users/cbf4568074bd483a8bcd1856c41f336d` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:57.986785 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dad4b7f-faec-42e4-a8bd-9440fdaa0559 None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:58.002279 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038cefc5-6157-4da0-a360-6cf413a73023 None admin] RBAC: Authorizing `identity:get_user(user_id=cbf4568074bd483a8bcd1856c41f336d)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:58.003916 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038cefc5-6157-4da0-a360-6cf413a73023 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:58.006392 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 516/1058] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Jul 27 10:35:57 2022] GET /identity/v3/users/cbf4568074bd483a8bcd1856c41f336d => generated 285 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 10:35:58.012106 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038cefc5-6157-4da0-a360-6cf413a73023 None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:58.082319 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:58.097335 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dad4b7f-faec-42e4-a8bd-9440fdaa0559 None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:58.117649 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038cefc5-6157-4da0-a360-6cf413a73023 None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:58.133262 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bc5e78-2727-460b-812c-e40afb62ae48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:58.134316 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bc5e78-2727-460b-812c-e40afb62ae48 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:58.134618 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bc5e78-2727-460b-812c-e40afb62ae48 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:58.135188 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bc5e78-2727-460b-812c-e40afb62ae48 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:58.145620 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79bc5e78-2727-460b-812c-e40afb62ae48 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 10:35:58.151013 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application [None req-79bc5e78-2727-460b-812c-e40afb62ae48 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 27 10:35:58.151013 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:35:58.151013 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:35:58.151013 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:35:58.151013 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:35:58.151013 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:35:58.151013 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:35:58.151013 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:35:58.151013 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:35:58.151013 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:35:58.151013 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:35:58.151013 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:35:58.151013 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 10:35:58.151013 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 10:35:58.151013 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 10:35:58.151013 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 10:35:58.151938 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 10:35:58.151938 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 10:35:58.151938 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 10:35:58.151938 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 10:35:58.151938 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:35:58.151938 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:35:58.151938 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:35:58.151938 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:35:58.151938 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:35:58.151938 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:35:58.151938 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:35:58.151938 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application with Lock( Jul 27 10:35:58.151938 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:35:58.151938 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:35:58.151938 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:35:58.151938 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:35:58.152770 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:35:58.152770 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:35:58.152770 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:35:58.152770 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:35:58.152770 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 10:35:58.152770 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 10:35:58.152770 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 10:35:58.152770 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 10:35:58.152770 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 10:35:58.152770 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 10:35:58.152770 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 27 10:35:58.152770 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Jul 27 10:35:58.152770 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 543/1059] 10.222.0.36 () {56 vars in 1120 bytes} [Wed Jul 27 10:35:57 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 10:35:58.154233 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904dead5-6c95-47df-aad2-cc249601f7e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:58.154898 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904dead5-6c95-47df-aad2-cc249601f7e3 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:58.155181 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904dead5-6c95-47df-aad2-cc249601f7e3 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:58.155455 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904dead5-6c95-47df-aad2-cc249601f7e3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:58.158226 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bc5e78-2727-460b-812c-e40afb62ae48 None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:58.167366 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:58.226243 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904dead5-6c95-47df-aad2-cc249601f7e3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:58.227634 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904dead5-6c95-47df-aad2-cc249601f7e3 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:58.261316 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bc5e78-2727-460b-812c-e40afb62ae48 None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:58.293575 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:58.297170 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea46df4e-eb3b-4cf8-a1f6-329b97d5d04e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:58.297789 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea46df4e-eb3b-4cf8-a1f6-329b97d5d04e None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:58.298070 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea46df4e-eb3b-4cf8-a1f6-329b97d5d04e None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:58.298341 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea46df4e-eb3b-4cf8-a1f6-329b97d5d04e None admin] PATH_INFO: `/v3/roles` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:58.299090 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea46df4e-eb3b-4cf8-a1f6-329b97d5d04e None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=86827) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:35:58.299395 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea46df4e-eb3b-4cf8-a1f6-329b97d5d04e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:58.300755 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea46df4e-eb3b-4cf8-a1f6-329b97d5d04e None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:58.309997 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 544/1060] 10.222.0.36 () {56 vars in 1115 bytes} [Wed Jul 27 10:35:58 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 10:35:58.315748 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496677a0-79ab-4cef-96cb-1b19e4d9e14d None None] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:58.316041 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496677a0-79ab-4cef-96cb-1b19e4d9e14d None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:58.316313 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496677a0-79ab-4cef-96cb-1b19e4d9e14d None None] PATH_INFO: `/` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:58.317544 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 545/1061] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 10:35:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 10:35:58.318801 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496677a0-79ab-4cef-96cb-1b19e4d9e14d None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:58.334555 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 517/1062] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jul 27 10:35:58 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 27 10:35:58.335812 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a67d82-5e8c-4abf-95c6-3fca52a270a1 None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:58.336104 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a67d82-5e8c-4abf-95c6-3fca52a270a1 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:58.336388 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a67d82-5e8c-4abf-95c6-3fca52a270a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:58.353853 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-75a67d82-5e8c-4abf-95c6-3fca52a270a1 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:35:58.355073 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-75a67d82-5e8c-4abf-95c6-3fca52a270a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:35:58.386302 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:58.420736 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496677a0-79ab-4cef-96cb-1b19e4d9e14d None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:58.421504 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 518/1063] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 10:35:58 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 10:35:58.422473 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a67d82-5e8c-4abf-95c6-3fca52a270a1 None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:58.458206 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885784d1-391f-4eee-a0cf-a592fa61c9f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:58.458901 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885784d1-391f-4eee-a0cf-a592fa61c9f8 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:58.459179 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885784d1-391f-4eee-a0cf-a592fa61c9f8 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:58.459452 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885784d1-391f-4eee-a0cf-a592fa61c9f8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:58.470420 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885784d1-391f-4eee-a0cf-a592fa61c9f8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:58.471901 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885784d1-391f-4eee-a0cf-a592fa61c9f8 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:58.486449 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application [None req-885784d1-391f-4eee-a0cf-a592fa61c9f8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 10:35:58.486449 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:35:58.486449 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:35:58.486449 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:35:58.486449 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:35:58.486449 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:35:58.486449 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:35:58.486449 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:35:58.486449 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:35:58.486449 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:35:58.486449 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:35:58.486449 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:35:58.486449 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 10:35:58.486449 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 10:35:58.486449 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 10:35:58.487692 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 10:35:58.487692 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:35:58.487692 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:35:58.487692 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:35:58.487692 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:35:58.487692 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:35:58.487692 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:35:58.487692 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:35:58.487692 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application with Lock( Jul 27 10:35:58.487692 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:35:58.487692 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:35:58.487692 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:35:58.487692 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:35:58.487692 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:35:58.487692 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:35:58.487692 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:35:58.487692 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:35:58.488710 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 10:35:58.488710 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 10:35:58.488710 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 10:35:58.488710 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Jul 27 10:35:58.488710 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 546/1064] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 10:35:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 10:35:58.488710 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885784d1-391f-4eee-a0cf-a592fa61c9f8 None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:58.523303 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:58.527872 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a67d82-5e8c-4abf-95c6-3fca52a270a1 None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:58.568140 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72f364c-d29a-495f-b2b0-6e75338b2b52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:58.568779 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72f364c-d29a-495f-b2b0-6e75338b2b52 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:58.569059 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72f364c-d29a-495f-b2b0-6e75338b2b52 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:58.569330 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72f364c-d29a-495f-b2b0-6e75338b2b52 None admin] PATH_INFO: `/v3/projects/c8790f7d27cf4ee38f3ff22cdbe26e5f` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:58.570468 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72f364c-d29a-495f-b2b0-6e75338b2b52 None admin] RBAC: Authorizing `identity:get_project(project_id=c8790f7d27cf4ee38f3ff22cdbe26e5f)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:58.571928 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72f364c-d29a-495f-b2b0-6e75338b2b52 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:58.573568 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 519/1065] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Jul 27 10:35:58 2022] GET /identity/v3/projects/c8790f7d27cf4ee38f3ff22cdbe26e5f => generated 311 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:35:58.578878 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72f364c-d29a-495f-b2b0-6e75338b2b52 None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:58.589503 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885784d1-391f-4eee-a0cf-a592fa61c9f8 None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:58.609735 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:58.624854 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05dfb42c-8ce5-4094-81a3-d4bc52da0223 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:58.625476 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dfb42c-8ce5-4094-81a3-d4bc52da0223 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:58.625755 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dfb42c-8ce5-4094-81a3-d4bc52da0223 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:58.626107 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dfb42c-8ce5-4094-81a3-d4bc52da0223 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:58.673608 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72f364c-d29a-495f-b2b0-6e75338b2b52 None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:58.693493 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05dfb42c-8ce5-4094-81a3-d4bc52da0223 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:58.694972 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05dfb42c-8ce5-4094-81a3-d4bc52da0223 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:58.714254 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621c0198-f3b3-45f5-a0fa-7c94d09730ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:58.714908 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621c0198-f3b3-45f5-a0fa-7c94d09730ee None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:58.715197 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621c0198-f3b3-45f5-a0fa-7c94d09730ee None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:58.715479 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621c0198-f3b3-45f5-a0fa-7c94d09730ee None admin] PATH_INFO: `/v3/domains` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:58.716171 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621c0198-f3b3-45f5-a0fa-7c94d09730ee None admin] RBAC: Adding query filter params (name=Default) {{(pid=86828) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:35:58.716469 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621c0198-f3b3-45f5-a0fa-7c94d09730ee None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:58.717836 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621c0198-f3b3-45f5-a0fa-7c94d09730ee None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:58.729673 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 520/1066] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 10:35:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:35:58.730450 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621c0198-f3b3-45f5-a0fa-7c94d09730ee None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:58.757125 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:58.807761 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 547/1067] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jul 27 10:35:58 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 27 10:35:58.808750 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05dfb42c-8ce5-4094-81a3-d4bc52da0223 None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:58.829430 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:58.829698 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621c0198-f3b3-45f5-a0fa-7c94d09730ee None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:58.864779 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c7dad1-eac0-451f-852d-0f54e72e1627 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:58.865408 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c7dad1-eac0-451f-852d-0f54e72e1627 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:58.865732 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c7dad1-eac0-451f-852d-0f54e72e1627 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:58.866004 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c7dad1-eac0-451f-852d-0f54e72e1627 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:58.866829 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c7dad1-eac0-451f-852d-0f54e72e1627 None admin] RBAC: Adding query filter params (role.id=21156560eb6b436bb567a40b4976f3f4, scope.project.id=c8790f7d27cf4ee38f3ff22cdbe26e5f, user.id=cbf4568074bd483a8bcd1856c41f336d) {{(pid=86828) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:35:58.867139 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c7dad1-eac0-451f-852d-0f54e72e1627 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:58.868563 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c7dad1-eac0-451f-852d-0f54e72e1627 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:58.873971 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 521/1068] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Jul 27 10:35:58 2022] GET /identity/v3/role_assignments?user.id=cbf4568074bd483a8bcd1856c41f336d&scope.project.id=c8790f7d27cf4ee38f3ff22cdbe26e5f&role.id=21156560eb6b436bb567a40b4976f3f4 => generated 259 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:35:58.874799 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c7dad1-eac0-451f-852d-0f54e72e1627 None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:58.915342 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05dfb42c-8ce5-4094-81a3-d4bc52da0223 None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:58.960111 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41819255-88ef-4e0e-a33d-1f24383e56c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:58.960772 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41819255-88ef-4e0e-a33d-1f24383e56c6 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:58.961052 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41819255-88ef-4e0e-a33d-1f24383e56c6 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:58.961315 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41819255-88ef-4e0e-a33d-1f24383e56c6 None admin] PATH_INFO: `/v3/users/nova` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:58.969264 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41819255-88ef-4e0e-a33d-1f24383e56c6 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:58.970865 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41819255-88ef-4e0e-a33d-1f24383e56c6 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:58.984812 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application [None req-41819255-88ef-4e0e-a33d-1f24383e56c6 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 27 10:35:58.984812 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:35:58.984812 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:35:58.984812 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:35:58.984812 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:35:58.984812 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:35:58.984812 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:35:58.984812 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:35:58.984812 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:35:58.984812 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:35:58.984812 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:35:58.984812 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:35:58.984812 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 10:35:58.984812 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 10:35:58.984812 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 10:35:58.984812 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 10:35:58.985852 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:35:58.985852 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:35:58.985852 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 10:35:58.985852 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 10:35:58.985852 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 10:35:58.985852 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 10:35:58.985852 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:35:58.985852 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:35:58.985852 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:35:58.985852 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:35:58.985852 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:35:58.985852 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application with Lock( Jul 27 10:35:58.985852 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:35:58.985852 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:35:58.985852 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:35:58.985852 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:35:58.985852 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:35:58.985852 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:35:58.986849 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:35:58.986849 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:35:58.986849 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 10:35:58.986849 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 10:35:58.986849 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 10:35:58.986849 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 10:35:58.986849 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 10:35:58.986849 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 10:35:58.986849 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 27 10:35:58.986849 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Jul 27 10:35:58.986849 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 548/1069] 10.222.0.36 () {56 vars in 1093 bytes} [Wed Jul 27 10:35:58 2022] GET /identity/v3/users/nova => generated 82 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 10:35:58.991976 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:58.992504 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41819255-88ef-4e0e-a33d-1f24383e56c6 None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:59.004023 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c7dad1-eac0-451f-852d-0f54e72e1627 None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:59.042144 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489bfb8d-94e2-4c6c-92e6-faddd036b4f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:59.042763 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489bfb8d-94e2-4c6c-92e6-faddd036b4f9 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:59.043047 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489bfb8d-94e2-4c6c-92e6-faddd036b4f9 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:59.043332 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489bfb8d-94e2-4c6c-92e6-faddd036b4f9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:59.055157 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489bfb8d-94e2-4c6c-92e6-faddd036b4f9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:59.056623 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489bfb8d-94e2-4c6c-92e6-faddd036b4f9 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:59.076823 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application [None req-489bfb8d-94e2-4c6c-92e6-faddd036b4f9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 10:35:59.076823 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:35:59.076823 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:35:59.076823 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:35:59.076823 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:35:59.076823 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:35:59.076823 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:35:59.076823 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:35:59.076823 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:35:59.076823 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:35:59.076823 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:35:59.076823 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:35:59.076823 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 10:35:59.076823 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 10:35:59.076823 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 10:35:59.077854 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 10:35:59.077854 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:35:59.077854 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:35:59.077854 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:35:59.077854 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:35:59.077854 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:35:59.077854 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:35:59.077854 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:35:59.077854 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application with Lock( Jul 27 10:35:59.077854 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:35:59.077854 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:35:59.077854 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:35:59.077854 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:35:59.077854 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:35:59.077854 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:35:59.077854 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:35:59.077854 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:35:59.078783 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 10:35:59.078783 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 10:35:59.078783 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 10:35:59.078783 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Jul 27 10:35:59.078783 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 522/1070] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 10:35:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 10:35:59.085532 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489bfb8d-94e2-4c6c-92e6-faddd036b4f9 None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:59.093218 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:59.114889 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41819255-88ef-4e0e-a33d-1f24383e56c6 None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:59.154376 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbde267-3c4c-47c5-988a-654207a132a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:59.155035 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbde267-3c4c-47c5-988a-654207a132a9 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:59.155318 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbde267-3c4c-47c5-988a-654207a132a9 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:59.155591 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbde267-3c4c-47c5-988a-654207a132a9 None admin] PATH_INFO: `/v3/users` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:59.156393 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbde267-3c4c-47c5-988a-654207a132a9 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86827) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:35:59.156709 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbde267-3c4c-47c5-988a-654207a132a9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:59.158148 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbde267-3c4c-47c5-988a-654207a132a9 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:59.176544 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 549/1071] 10.222.0.36 () {56 vars in 1133 bytes} [Wed Jul 27 10:35:58 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 10:35:59.182125 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbde267-3c4c-47c5-988a-654207a132a9 None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:59.194433 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489bfb8d-94e2-4c6c-92e6-faddd036b4f9 None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:59.231725 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615ee56d-f98a-4283-8932-66318a605e14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:59.232358 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615ee56d-f98a-4283-8932-66318a605e14 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:59.232638 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615ee56d-f98a-4283-8932-66318a605e14 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:59.232914 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615ee56d-f98a-4283-8932-66318a605e14 None admin] PATH_INFO: `/v3/domains` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:59.233606 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615ee56d-f98a-4283-8932-66318a605e14 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86828) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:35:59.233911 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615ee56d-f98a-4283-8932-66318a605e14 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:59.235361 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615ee56d-f98a-4283-8932-66318a605e14 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:59.247359 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 523/1072] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 10:35:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 10:35:59.292051 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbde267-3c4c-47c5-988a-654207a132a9 None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:59.331385 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274fde1e-bcc6-4929-a8b6-d0408f830cb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:59.332064 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274fde1e-bcc6-4929-a8b6-d0408f830cb8 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:59.332348 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274fde1e-bcc6-4929-a8b6-d0408f830cb8 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:59.332625 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274fde1e-bcc6-4929-a8b6-d0408f830cb8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:59.340726 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:59.346342 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274fde1e-bcc6-4929-a8b6-d0408f830cb8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:59.347887 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274fde1e-bcc6-4929-a8b6-d0408f830cb8 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:59.363958 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application [None req-274fde1e-bcc6-4929-a8b6-d0408f830cb8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 10:35:59.363958 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:35:59.363958 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:35:59.363958 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:35:59.363958 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:35:59.363958 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:35:59.363958 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:35:59.363958 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:35:59.363958 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:35:59.363958 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:35:59.363958 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:35:59.363958 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:35:59.363958 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 10:35:59.363958 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 10:35:59.363958 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 10:35:59.364942 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 10:35:59.364942 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:35:59.364942 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:35:59.364942 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:35:59.364942 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:35:59.364942 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:35:59.364942 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:35:59.364942 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:35:59.364942 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application with Lock( Jul 27 10:35:59.364942 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:35:59.364942 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:35:59.364942 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:35:59.364942 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:35:59.364942 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:35:59.364942 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:35:59.364942 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:35:59.364942 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:35:59.365830 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 10:35:59.365830 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 10:35:59.365830 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 10:35:59.365830 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Jul 27 10:35:59.365830 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 550/1073] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 10:35:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 10:35:59.372951 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615ee56d-f98a-4283-8932-66318a605e14 None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:59.376548 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:59.489603 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615ee56d-f98a-4283-8932-66318a605e14 None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:59.526293 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55489b2e-b9c1-40f6-a458-661c9c94de39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:59.526959 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55489b2e-b9c1-40f6-a458-661c9c94de39 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:59.527243 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55489b2e-b9c1-40f6-a458-661c9c94de39 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:59.527521 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55489b2e-b9c1-40f6-a458-661c9c94de39 None admin] PATH_INFO: `/v3/domains` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:59.528221 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55489b2e-b9c1-40f6-a458-661c9c94de39 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86828) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:35:59.528528 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55489b2e-b9c1-40f6-a458-661c9c94de39 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:59.529904 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55489b2e-b9c1-40f6-a458-661c9c94de39 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:59.541730 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 524/1074] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 10:35:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:35:59.547684 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274fde1e-bcc6-4929-a8b6-d0408f830cb8 None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:59.649497 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274fde1e-bcc6-4929-a8b6-d0408f830cb8 None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:59.686891 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d52f5b6-3360-458c-8f62-0cae5d66a575 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:59.687531 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d52f5b6-3360-458c-8f62-0cae5d66a575 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:59.687833 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d52f5b6-3360-458c-8f62-0cae5d66a575 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:59.688109 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d52f5b6-3360-458c-8f62-0cae5d66a575 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:59.695243 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d52f5b6-3360-458c-8f62-0cae5d66a575 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:59.696690 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d52f5b6-3360-458c-8f62-0cae5d66a575 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:59.707826 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application [None req-6d52f5b6-3360-458c-8f62-0cae5d66a575 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 10:35:59.707826 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:35:59.707826 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:35:59.707826 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:35:59.707826 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:35:59.707826 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:35:59.707826 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:35:59.707826 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:35:59.707826 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:35:59.707826 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:35:59.707826 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:35:59.707826 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:35:59.707826 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 10:35:59.707826 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 10:35:59.707826 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 10:35:59.708690 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 10:35:59.708690 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:35:59.708690 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:35:59.708690 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:35:59.708690 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:35:59.708690 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:35:59.708690 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:35:59.708690 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:35:59.708690 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application with Lock( Jul 27 10:35:59.708690 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:35:59.708690 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:35:59.708690 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:35:59.708690 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:35:59.708690 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:35:59.708690 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:35:59.708690 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:35:59.708690 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:35:59.709627 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 10:35:59.709627 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 10:35:59.709627 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 10:35:59.709627 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 10:35:59.709627 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 10:35:59.709627 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 10:35:59.709627 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 10:35:59.709627 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Jul 27 10:35:59.709627 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 551/1075] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Jul 27 10:35:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 10:35:59.714845 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55489b2e-b9c1-40f6-a458-661c9c94de39 None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:59.774511 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:59.812365 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55489b2e-b9c1-40f6-a458-661c9c94de39 None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:35:59.845191 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9969e0-3db1-41c5-ac39-23db20828e41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:35:59.845806 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9969e0-3db1-41c5-ac39-23db20828e41 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:35:59.846088 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9969e0-3db1-41c5-ac39-23db20828e41 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:35:59.846365 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9969e0-3db1-41c5-ac39-23db20828e41 None admin] PATH_INFO: `/v3/projects` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:35:59.847138 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9969e0-3db1-41c5-ac39-23db20828e41 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86828) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:35:59.847448 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9969e0-3db1-41c5-ac39-23db20828e41 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:35:59.848869 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9969e0-3db1-41c5-ac39-23db20828e41 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:35:59.861062 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 525/1076] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Jul 27 10:35:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:35:59.867576 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d52f5b6-3360-458c-8f62-0cae5d66a575 None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:35:59.916274 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:35:59.972494 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d52f5b6-3360-458c-8f62-0cae5d66a575 None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:36:00.008164 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825574fe-1ef5-486c-9859-4eacf6ea8720 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:36:00.009105 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825574fe-1ef5-486c-9859-4eacf6ea8720 None admin] REQUEST_METHOD: `PUT` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:36:00.009385 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825574fe-1ef5-486c-9859-4eacf6ea8720 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:36:00.009660 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825574fe-1ef5-486c-9859-4eacf6ea8720 None admin] PATH_INFO: `/v3/projects/5f296d8e3b8f4001909addc49d8dac01/users/e059d8b6c28c4fcba925d05d7c98e4ac/roles/722097dd0463434b9c4ecaebe7ecf9ea` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:36:00.011858 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825574fe-1ef5-486c-9859-4eacf6ea8720 None admin] RBAC: Authorizing `identity:create_grant(project_id=5f296d8e3b8f4001909addc49d8dac01, user_id=e059d8b6c28c4fcba925d05d7c98e4ac, role_id=722097dd0463434b9c4ecaebe7ecf9ea)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:36:00.013286 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825574fe-1ef5-486c-9859-4eacf6ea8720 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:36:00.026397 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 552/1077] 10.222.0.36 () {58 vars in 1439 bytes} [Wed Jul 27 10:35:59 2022] PUT /identity/v3/projects/5f296d8e3b8f4001909addc49d8dac01/users/e059d8b6c28c4fcba925d05d7c98e4ac/roles/722097dd0463434b9c4ecaebe7ecf9ea => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 10:36:00.895803 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975fef3d-f4f4-47db-8e61-3cb98be1d11b None None] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:36:00.896925 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975fef3d-f4f4-47db-8e61-3cb98be1d11b None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:36:00.896925 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975fef3d-f4f4-47db-8e61-3cb98be1d11b None None] PATH_INFO: `/` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:36:00.898291 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 526/1078] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 10:36:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 10:36:00.903519 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e05b20c-2bbc-4a87-b82b-fc0046ce486b None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:36:00.903896 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e05b20c-2bbc-4a87-b82b-fc0046ce486b None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:36:00.904210 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e05b20c-2bbc-4a87-b82b-fc0046ce486b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:36:00.922066 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-4e05b20c-2bbc-4a87-b82b-fc0046ce486b None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:36:00.923280 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-4e05b20c-2bbc-4a87-b82b-fc0046ce486b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:36:00.992270 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 553/1079] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 10:36:00 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 10:36:01.017188 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975fef3d-f4f4-47db-8e61-3cb98be1d11b None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:36:01.134580 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975fef3d-f4f4-47db-8e61-3cb98be1d11b None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:36:01.157653 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:36:01.169521 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c59e5f1-cc4e-434b-bd77-ebfaf6aa2fb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:36:01.170162 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c59e5f1-cc4e-434b-bd77-ebfaf6aa2fb4 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:36:01.170447 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c59e5f1-cc4e-434b-bd77-ebfaf6aa2fb4 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:36:01.170781 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c59e5f1-cc4e-434b-bd77-ebfaf6aa2fb4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:36:01.181590 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c59e5f1-cc4e-434b-bd77-ebfaf6aa2fb4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 10:36:01.186519 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application [None req-3c59e5f1-cc4e-434b-bd77-ebfaf6aa2fb4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 10:36:01.186519 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:36:01.186519 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:36:01.186519 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:36:01.186519 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:36:01.186519 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:36:01.186519 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:36:01.186519 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:36:01.186519 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:36:01.186519 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:36:01.186519 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:36:01.186519 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:36:01.186519 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 10:36:01.186519 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 10:36:01.186519 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 10:36:01.186519 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 10:36:01.187609 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 10:36:01.187609 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 10:36:01.187609 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 10:36:01.187609 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 10:36:01.187609 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:36:01.187609 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:36:01.187609 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:36:01.187609 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:36:01.187609 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:36:01.187609 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:36:01.187609 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:36:01.187609 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application with Lock( Jul 27 10:36:01.187609 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:36:01.187609 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:36:01.187609 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:36:01.187609 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:36:01.188445 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:36:01.188445 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:36:01.188445 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:36:01.188445 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:36:01.188445 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 10:36:01.188445 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 10:36:01.188445 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 10:36:01.188445 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 10:36:01.188445 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 10:36:01.188445 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 10:36:01.188445 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 10:36:01.188445 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Jul 27 10:36:01.188445 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 527/1080] 10.222.0.36 () {56 vars in 1096 bytes} [Wed Jul 27 10:36:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 10:36:01.194301 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e05b20c-2bbc-4a87-b82b-fc0046ce486b None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:36:01.234312 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:36:01.294891 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e05b20c-2bbc-4a87-b82b-fc0046ce486b None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:36:01.328563 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b7a7a5-1d2f-4dfa-82ca-04cc7bb9b773 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:36:01.329183 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b7a7a5-1d2f-4dfa-82ca-04cc7bb9b773 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:36:01.329462 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b7a7a5-1d2f-4dfa-82ca-04cc7bb9b773 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:36:01.329737 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b7a7a5-1d2f-4dfa-82ca-04cc7bb9b773 None admin] PATH_INFO: `/v3/roles` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:36:01.330452 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b7a7a5-1d2f-4dfa-82ca-04cc7bb9b773 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86827) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:36:01.330798 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b7a7a5-1d2f-4dfa-82ca-04cc7bb9b773 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:36:01.332161 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b7a7a5-1d2f-4dfa-82ca-04cc7bb9b773 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:36:01.340029 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 554/1081] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jul 27 10:36:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:36:01.345699 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c59e5f1-cc4e-434b-bd77-ebfaf6aa2fb4 None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:36:01.447416 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c59e5f1-cc4e-434b-bd77-ebfaf6aa2fb4 None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:36:01.480311 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad29fa2-5807-4ab5-98c8-b7b50a4a6ec9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:36:01.480958 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad29fa2-5807-4ab5-98c8-b7b50a4a6ec9 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:36:01.481243 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad29fa2-5807-4ab5-98c8-b7b50a4a6ec9 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:36:01.481522 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad29fa2-5807-4ab5-98c8-b7b50a4a6ec9 None admin] PATH_INFO: `/v3/users/cbf4568074bd483a8bcd1856c41f336d` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:36:01.482710 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad29fa2-5807-4ab5-98c8-b7b50a4a6ec9 None admin] RBAC: Authorizing `identity:get_user(user_id=cbf4568074bd483a8bcd1856c41f336d)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:36:01.484156 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad29fa2-5807-4ab5-98c8-b7b50a4a6ec9 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:36:01.485795 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 528/1082] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Jul 27 10:36:01 2022] GET /identity/v3/users/cbf4568074bd483a8bcd1856c41f336d => generated 285 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:36:01.492204 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b7a7a5-1d2f-4dfa-82ca-04cc7bb9b773 None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:36:01.590913 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b7a7a5-1d2f-4dfa-82ca-04cc7bb9b773 None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:36:01.592540 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:36:01.626518 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7053c5f4-c7f5-445c-93fa-8026dd5bd49c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:36:01.627192 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7053c5f4-c7f5-445c-93fa-8026dd5bd49c None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:36:01.627497 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7053c5f4-c7f5-445c-93fa-8026dd5bd49c None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:36:01.627778 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7053c5f4-c7f5-445c-93fa-8026dd5bd49c None admin] PATH_INFO: `/v3/projects/c8790f7d27cf4ee38f3ff22cdbe26e5f` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:36:01.628937 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7053c5f4-c7f5-445c-93fa-8026dd5bd49c None admin] RBAC: Authorizing `identity:get_project(project_id=c8790f7d27cf4ee38f3ff22cdbe26e5f)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:36:01.630393 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7053c5f4-c7f5-445c-93fa-8026dd5bd49c None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:36:01.632108 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 555/1083] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Jul 27 10:36:01 2022] GET /identity/v3/projects/c8790f7d27cf4ee38f3ff22cdbe26e5f => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 10:36:01.638284 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad29fa2-5807-4ab5-98c8-b7b50a4a6ec9 None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:36:01.687495 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:36:01.737132 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad29fa2-5807-4ab5-98c8-b7b50a4a6ec9 None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:36:01.769642 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c7395b-0cae-4257-9285-da968359d3d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:36:01.770318 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c7395b-0cae-4257-9285-da968359d3d4 None admin] REQUEST_METHOD: `PUT` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:36:01.770619 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c7395b-0cae-4257-9285-da968359d3d4 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:36:01.771429 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c7395b-0cae-4257-9285-da968359d3d4 None admin] PATH_INFO: `/v3/projects/c8790f7d27cf4ee38f3ff22cdbe26e5f/users/cbf4568074bd483a8bcd1856c41f336d/roles/21156560eb6b436bb567a40b4976f3f4` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:36:01.773377 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c7395b-0cae-4257-9285-da968359d3d4 None admin] RBAC: Authorizing `identity:create_grant(project_id=c8790f7d27cf4ee38f3ff22cdbe26e5f, user_id=cbf4568074bd483a8bcd1856c41f336d, role_id=21156560eb6b436bb567a40b4976f3f4)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:36:01.774837 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c7395b-0cae-4257-9285-da968359d3d4 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:36:01.790082 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 529/1084] 10.222.0.36 () {58 vars in 1439 bytes} [Wed Jul 27 10:36:01 2022] PUT /identity/v3/projects/c8790f7d27cf4ee38f3ff22cdbe26e5f/users/cbf4568074bd483a8bcd1856c41f336d/roles/21156560eb6b436bb567a40b4976f3f4 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 10:36:02.014637 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0daa062f-033f-4cb9-9da0-1e4b34b8b885 None None] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:36:02.015154 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0daa062f-033f-4cb9-9da0-1e4b34b8b885 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:36:02.015154 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0daa062f-033f-4cb9-9da0-1e4b34b8b885 None None] PATH_INFO: `/` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:36:02.016484 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 556/1085] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 10:36:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 10:36:02.021739 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66637fb-332a-4afd-b987-51b8d1b5005e None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:36:02.022175 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66637fb-332a-4afd-b987-51b8d1b5005e None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:36:02.022368 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66637fb-332a-4afd-b987-51b8d1b5005e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:36:02.040251 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-e66637fb-332a-4afd-b987-51b8d1b5005e None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:36:02.041429 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-e66637fb-332a-4afd-b987-51b8d1b5005e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:36:02.109355 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 530/1086] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 10:36:02 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 10:36:02.132776 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0daa062f-033f-4cb9-9da0-1e4b34b8b885 None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:36:02.233471 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0daa062f-033f-4cb9-9da0-1e4b34b8b885 None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:36:02.266933 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5227dd5e-36e5-4aea-b97d-371ba267a7c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:36:02.267510 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5227dd5e-36e5-4aea-b97d-371ba267a7c2 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:36:02.267793 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5227dd5e-36e5-4aea-b97d-371ba267a7c2 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:36:02.268065 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5227dd5e-36e5-4aea-b97d-371ba267a7c2 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:36:02.278388 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5227dd5e-36e5-4aea-b97d-371ba267a7c2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 10:36:02.283367 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application [None req-5227dd5e-36e5-4aea-b97d-371ba267a7c2 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 27 10:36:02.283367 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:36:02.283367 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:36:02.283367 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:36:02.283367 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:36:02.283367 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:36:02.283367 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:36:02.283367 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:36:02.283367 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:36:02.283367 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:36:02.283367 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:36:02.283367 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:36:02.283367 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 10:36:02.283367 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 10:36:02.283367 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 10:36:02.283367 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 10:36:02.284536 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 10:36:02.284536 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 10:36:02.284536 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 10:36:02.284536 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 10:36:02.284536 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:36:02.284536 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:36:02.284536 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:36:02.284536 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:36:02.284536 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:36:02.284536 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:36:02.284536 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:36:02.284536 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application with Lock( Jul 27 10:36:02.284536 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:36:02.284536 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:36:02.284536 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:36:02.284536 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:36:02.285409 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:36:02.285409 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:36:02.285409 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:36:02.285409 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:36:02.285409 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 10:36:02.285409 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 10:36:02.285409 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 10:36:02.285409 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 10:36:02.285409 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 10:36:02.285409 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 10:36:02.285409 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 27 10:36:02.285409 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Jul 27 10:36:02.285409 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 557/1087] 10.222.0.36 () {56 vars in 1120 bytes} [Wed Jul 27 10:36:02 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 10:36:02.291115 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66637fb-332a-4afd-b987-51b8d1b5005e None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:36:02.331311 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:36:02.388051 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66637fb-332a-4afd-b987-51b8d1b5005e None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:36:02.421294 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47f8650-4b0a-4a4a-b13b-648266968ba8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:36:02.421908 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47f8650-4b0a-4a4a-b13b-648266968ba8 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:36:02.422217 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47f8650-4b0a-4a4a-b13b-648266968ba8 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:36:02.422494 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47f8650-4b0a-4a4a-b13b-648266968ba8 None admin] PATH_INFO: `/v3/roles` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:36:02.423339 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47f8650-4b0a-4a4a-b13b-648266968ba8 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=86828) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:36:02.423643 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47f8650-4b0a-4a4a-b13b-648266968ba8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:36:02.425012 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47f8650-4b0a-4a4a-b13b-648266968ba8 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:36:02.438070 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 531/1088] 10.222.0.36 () {56 vars in 1115 bytes} [Wed Jul 27 10:36:02 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:36:02.439837 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5227dd5e-36e5-4aea-b97d-371ba267a7c2 None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:36:02.470996 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:36:02.541265 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5227dd5e-36e5-4aea-b97d-371ba267a7c2 None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:36:02.573453 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d69e01-f2dc-4d74-957a-747a812be2a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:36:02.574087 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d69e01-f2dc-4d74-957a-747a812be2a2 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:36:02.574371 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d69e01-f2dc-4d74-957a-747a812be2a2 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:36:02.574678 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d69e01-f2dc-4d74-957a-747a812be2a2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:36:02.585484 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d69e01-f2dc-4d74-957a-747a812be2a2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:36:02.586965 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d69e01-f2dc-4d74-957a-747a812be2a2 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:36:02.601578 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application [None req-86d69e01-f2dc-4d74-957a-747a812be2a2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 10:36:02.601578 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:36:02.601578 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:36:02.601578 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:36:02.601578 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:36:02.601578 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:36:02.601578 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:36:02.601578 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:36:02.601578 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:36:02.601578 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:36:02.601578 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:36:02.601578 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:36:02.601578 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 10:36:02.601578 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 10:36:02.601578 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 10:36:02.602487 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 10:36:02.602487 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:36:02.602487 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:36:02.602487 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:36:02.602487 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:36:02.602487 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:36:02.602487 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:36:02.602487 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:36:02.602487 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application with Lock( Jul 27 10:36:02.602487 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:36:02.602487 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:36:02.602487 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:36:02.602487 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:36:02.602487 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:36:02.602487 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:36:02.602487 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:36:02.602487 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:36:02.603461 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 10:36:02.603461 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 10:36:02.603461 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 10:36:02.603461 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Jul 27 10:36:02.603461 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 558/1089] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 10:36:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 10:36:02.608502 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47f8650-4b0a-4a4a-b13b-648266968ba8 None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:36:02.705544 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47f8650-4b0a-4a4a-b13b-648266968ba8 None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:36:02.707082 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:36:02.740746 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726b5492-3ca3-4703-b018-ff64502b0c56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:36:02.741366 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726b5492-3ca3-4703-b018-ff64502b0c56 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:36:02.741653 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726b5492-3ca3-4703-b018-ff64502b0c56 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:36:02.741930 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726b5492-3ca3-4703-b018-ff64502b0c56 None admin] PATH_INFO: `/v3/domains` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:36:02.742644 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726b5492-3ca3-4703-b018-ff64502b0c56 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86828) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:36:02.742960 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726b5492-3ca3-4703-b018-ff64502b0c56 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:36:02.744321 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726b5492-3ca3-4703-b018-ff64502b0c56 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:36:02.755880 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 532/1090] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 10:36:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:36:02.761713 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d69e01-f2dc-4d74-957a-747a812be2a2 None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:36:02.842448 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:36:02.860504 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d69e01-f2dc-4d74-957a-747a812be2a2 None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:36:02.895560 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6adebd7-0479-4446-9cdf-95398f443034 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:36:02.896804 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6adebd7-0479-4446-9cdf-95398f443034 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:36:02.897285 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6adebd7-0479-4446-9cdf-95398f443034 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:36:02.897771 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6adebd7-0479-4446-9cdf-95398f443034 None admin] PATH_INFO: `/v3/users/nova` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:36:02.906848 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6adebd7-0479-4446-9cdf-95398f443034 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:36:02.908290 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6adebd7-0479-4446-9cdf-95398f443034 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:36:02.919376 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application [None req-d6adebd7-0479-4446-9cdf-95398f443034 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 27 10:36:02.919376 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:36:02.919376 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:36:02.919376 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:36:02.919376 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:36:02.919376 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:36:02.919376 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:36:02.919376 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:36:02.919376 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:36:02.919376 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:36:02.919376 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:36:02.919376 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:36:02.919376 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 10:36:02.919376 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 10:36:02.919376 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 10:36:02.919376 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 10:36:02.920331 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:36:02.920331 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:36:02.920331 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 10:36:02.920331 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 10:36:02.920331 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 10:36:02.920331 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 10:36:02.920331 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:36:02.920331 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:36:02.920331 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:36:02.920331 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:36:02.920331 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:36:02.920331 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application with Lock( Jul 27 10:36:02.920331 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:36:02.920331 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:36:02.920331 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:36:02.920331 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:36:02.920331 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:36:02.920331 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:36:02.921281 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:36:02.921281 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:36:02.921281 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 10:36:02.921281 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 10:36:02.921281 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 10:36:02.921281 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 10:36:02.921281 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 10:36:02.921281 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 10:36:02.921281 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 27 10:36:02.921281 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Jul 27 10:36:02.921281 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 559/1091] 10.222.0.36 () {56 vars in 1093 bytes} [Wed Jul 27 10:36:02 2022] GET /identity/v3/users/nova => generated 82 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 10:36:02.926427 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726b5492-3ca3-4703-b018-ff64502b0c56 None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:36:03.025131 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726b5492-3ca3-4703-b018-ff64502b0c56 None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:36:03.057571 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d94a1c-177b-4f1d-860d-ab1c461e4ce7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:36:03.058190 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d94a1c-177b-4f1d-860d-ab1c461e4ce7 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:36:03.058475 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d94a1c-177b-4f1d-860d-ab1c461e4ce7 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:36:03.058783 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d94a1c-177b-4f1d-860d-ab1c461e4ce7 None admin] PATH_INFO: `/v3/users` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:36:03.059578 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d94a1c-177b-4f1d-860d-ab1c461e4ce7 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86828) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:36:03.059887 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d94a1c-177b-4f1d-860d-ab1c461e4ce7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:36:03.061315 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d94a1c-177b-4f1d-860d-ab1c461e4ce7 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:36:03.078265 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 533/1092] 10.222.0.36 () {56 vars in 1133 bytes} [Wed Jul 27 10:36:02 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:36:03.084494 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6adebd7-0479-4446-9cdf-95398f443034 None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:36:03.182707 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6adebd7-0479-4446-9cdf-95398f443034 None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:36:03.216035 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1dd004-5927-4ce3-b0b5-3e235c3ef6ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:36:03.216640 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1dd004-5927-4ce3-b0b5-3e235c3ef6ca None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:36:03.216917 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1dd004-5927-4ce3-b0b5-3e235c3ef6ca None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:36:03.217186 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1dd004-5927-4ce3-b0b5-3e235c3ef6ca None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:36:03.228945 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:36:03.283865 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1dd004-5927-4ce3-b0b5-3e235c3ef6ca None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:36:03.285312 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1dd004-5927-4ce3-b0b5-3e235c3ef6ca None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:36:03.382376 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 560/1093] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jul 27 10:36:03 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 27 10:36:03.388499 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d94a1c-177b-4f1d-860d-ab1c461e4ce7 None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:36:03.409568 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:36:03.486374 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d94a1c-177b-4f1d-860d-ab1c461e4ce7 None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:36:03.520270 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a2653a-5866-45f0-b58a-8b7a6df62bc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:36:03.520940 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a2653a-5866-45f0-b58a-8b7a6df62bc5 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:36:03.521225 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a2653a-5866-45f0-b58a-8b7a6df62bc5 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:36:03.521499 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a2653a-5866-45f0-b58a-8b7a6df62bc5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:36:03.531394 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a2653a-5866-45f0-b58a-8b7a6df62bc5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:36:03.532837 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a2653a-5866-45f0-b58a-8b7a6df62bc5 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:36:03.547148 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application [None req-68a2653a-5866-45f0-b58a-8b7a6df62bc5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 10:36:03.547148 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:36:03.547148 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:36:03.547148 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:36:03.547148 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:36:03.547148 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:36:03.547148 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:36:03.547148 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:36:03.547148 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:36:03.547148 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:36:03.547148 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:36:03.547148 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:36:03.547148 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 10:36:03.547148 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 10:36:03.547148 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 10:36:03.548025 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 10:36:03.548025 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:36:03.548025 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:36:03.548025 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:36:03.548025 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:36:03.548025 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:36:03.548025 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:36:03.548025 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:36:03.548025 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application with Lock( Jul 27 10:36:03.548025 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:36:03.548025 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:36:03.548025 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:36:03.548025 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:36:03.548025 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:36:03.548025 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:36:03.548025 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:36:03.548025 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:36:03.549055 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 10:36:03.549055 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 10:36:03.549055 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 10:36:03.549055 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Jul 27 10:36:03.549055 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 534/1094] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 10:36:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 10:36:03.554487 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1dd004-5927-4ce3-b0b5-3e235c3ef6ca None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:36:03.596015 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:36:03.652292 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1dd004-5927-4ce3-b0b5-3e235c3ef6ca None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:36:03.685615 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f545f919-85de-45b8-b019-73a2aeba72f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:36:03.686231 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f545f919-85de-45b8-b019-73a2aeba72f9 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:36:03.686513 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f545f919-85de-45b8-b019-73a2aeba72f9 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:36:03.686816 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f545f919-85de-45b8-b019-73a2aeba72f9 None admin] PATH_INFO: `/v3/domains` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:36:03.687506 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f545f919-85de-45b8-b019-73a2aeba72f9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86827) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:36:03.687812 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f545f919-85de-45b8-b019-73a2aeba72f9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:36:03.689184 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f545f919-85de-45b8-b019-73a2aeba72f9 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:36:03.700192 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 561/1095] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 10:36:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:36:03.705891 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a2653a-5866-45f0-b58a-8b7a6df62bc5 None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:36:03.769218 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d16cad8-e0c3-49af-812a-927bca13c392 None None] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:36:03.769419 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d16cad8-e0c3-49af-812a-927bca13c392 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:36:03.769683 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d16cad8-e0c3-49af-812a-927bca13c392 None None] PATH_INFO: `/` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:36:03.771131 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 562/1096] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 10:36:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 10:36:03.775532 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:36:03.775959 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9abe730-7213-4a79-9c23-e42ffc8a929b None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:36:03.776212 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9abe730-7213-4a79-9c23-e42ffc8a929b None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:36:03.776490 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9abe730-7213-4a79-9c23-e42ffc8a929b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:36:03.794499 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-e9abe730-7213-4a79-9c23-e42ffc8a929b None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:36:03.795739 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-e9abe730-7213-4a79-9c23-e42ffc8a929b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:36:03.806542 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a2653a-5866-45f0-b58a-8b7a6df62bc5 None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:36:03.839233 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1014bf12-7c9a-4066-8f8e-a34d92167597 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:36:03.839874 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1014bf12-7c9a-4066-8f8e-a34d92167597 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:36:03.840156 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1014bf12-7c9a-4066-8f8e-a34d92167597 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:36:03.840431 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1014bf12-7c9a-4066-8f8e-a34d92167597 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:36:03.847018 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1014bf12-7c9a-4066-8f8e-a34d92167597 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:36:03.848460 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1014bf12-7c9a-4066-8f8e-a34d92167597 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:36:03.860124 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application [None req-1014bf12-7c9a-4066-8f8e-a34d92167597 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 10:36:03.860124 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:36:03.860124 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:36:03.860124 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:36:03.860124 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:36:03.860124 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:36:03.860124 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:36:03.860124 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:36:03.860124 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:36:03.860124 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:36:03.860124 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:36:03.860124 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:36:03.860124 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 10:36:03.860124 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 10:36:03.860124 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 10:36:03.861235 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 10:36:03.861235 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:36:03.861235 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:36:03.861235 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:36:03.861235 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:36:03.861235 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:36:03.861235 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:36:03.861235 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:36:03.861235 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application with Lock( Jul 27 10:36:03.861235 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:36:03.861235 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:36:03.861235 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:36:03.861235 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:36:03.861235 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:36:03.861235 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:36:03.861235 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:36:03.861235 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:36:03.862374 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 10:36:03.862374 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 10:36:03.862374 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 10:36:03.862374 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 10:36:03.862374 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 10:36:03.862374 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 10:36:03.862374 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 10:36:03.862374 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Jul 27 10:36:03.862374 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 535/1097] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Jul 27 10:36:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 10:36:03.864426 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 563/1098] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 10:36:03 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 27 10:36:03.867603 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1014bf12-7c9a-4066-8f8e-a34d92167597 None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:36:03.887605 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9abe730-7213-4a79-9c23-e42ffc8a929b None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:36:03.918114 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:36:03.968249 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1014bf12-7c9a-4066-8f8e-a34d92167597 None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:36:03.988762 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9abe730-7213-4a79-9c23-e42ffc8a929b None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:36:03.997022 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:36:04.002887 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46e372e-afd1-4266-9fe3-0602b6656547 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:36:04.003512 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46e372e-afd1-4266-9fe3-0602b6656547 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:36:04.003799 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46e372e-afd1-4266-9fe3-0602b6656547 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:36:04.004079 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46e372e-afd1-4266-9fe3-0602b6656547 None admin] PATH_INFO: `/v3/projects` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:36:04.004832 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46e372e-afd1-4266-9fe3-0602b6656547 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86828) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:36:04.005168 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46e372e-afd1-4266-9fe3-0602b6656547 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:36:04.006659 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46e372e-afd1-4266-9fe3-0602b6656547 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:36:04.018807 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 536/1099] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Jul 27 10:36:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 10:36:04.025372 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46e372e-afd1-4266-9fe3-0602b6656547 None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:36:04.026746 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953a23e3-56ee-451f-a458-9d3bf573a75b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:36:04.027397 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953a23e3-56ee-451f-a458-9d3bf573a75b None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:36:04.027683 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953a23e3-56ee-451f-a458-9d3bf573a75b None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:36:04.027962 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953a23e3-56ee-451f-a458-9d3bf573a75b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:36:04.038690 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953a23e3-56ee-451f-a458-9d3bf573a75b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 10:36:04.043611 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application [None req-953a23e3-56ee-451f-a458-9d3bf573a75b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 10:36:04.043611 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:36:04.043611 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:36:04.043611 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:36:04.043611 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:36:04.043611 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:36:04.043611 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:36:04.043611 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:36:04.043611 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:36:04.043611 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:36:04.043611 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:36:04.043611 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:36:04.043611 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 10:36:04.043611 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 10:36:04.043611 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 10:36:04.043611 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 10:36:04.044737 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 10:36:04.044737 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 10:36:04.044737 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 10:36:04.044737 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 10:36:04.044737 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:36:04.044737 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:36:04.044737 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:36:04.044737 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:36:04.044737 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:36:04.044737 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:36:04.044737 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:36:04.044737 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application with Lock( Jul 27 10:36:04.044737 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:36:04.044737 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:36:04.044737 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:36:04.044737 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:36:04.051171 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:36:04.051171 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:36:04.051171 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:36:04.051171 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:36:04.051171 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 10:36:04.051171 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 10:36:04.051171 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 10:36:04.051171 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 10:36:04.051171 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 10:36:04.051171 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 10:36:04.051171 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 10:36:04.051171 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Jul 27 10:36:04.051171 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 564/1100] 10.222.0.36 () {56 vars in 1096 bytes} [Wed Jul 27 10:36:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 10:36:04.051171 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953a23e3-56ee-451f-a458-9d3bf573a75b None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:36:04.122840 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46e372e-afd1-4266-9fe3-0602b6656547 None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:36:04.141694 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:36:04.152039 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953a23e3-56ee-451f-a458-9d3bf573a75b None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:36:04.156833 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd18d1d6-85fd-44f5-a220-7673009c37d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:36:04.157449 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd18d1d6-85fd-44f5-a220-7673009c37d3 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:36:04.157728 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd18d1d6-85fd-44f5-a220-7673009c37d3 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:36:04.157997 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd18d1d6-85fd-44f5-a220-7673009c37d3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:36:04.158812 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd18d1d6-85fd-44f5-a220-7673009c37d3 None admin] RBAC: Adding query filter params (role.id=722097dd0463434b9c4ecaebe7ecf9ea, scope.project.id=5f296d8e3b8f4001909addc49d8dac01, user.id=e059d8b6c28c4fcba925d05d7c98e4ac) {{(pid=86828) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:36:04.159120 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd18d1d6-85fd-44f5-a220-7673009c37d3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:36:04.160534 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd18d1d6-85fd-44f5-a220-7673009c37d3 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:36:04.167201 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 537/1101] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Jul 27 10:36:04 2022] GET /identity/v3/role_assignments?user.id=e059d8b6c28c4fcba925d05d7c98e4ac&scope.project.id=5f296d8e3b8f4001909addc49d8dac01&role.id=722097dd0463434b9c4ecaebe7ecf9ea => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 10:36:04.187325 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da19df1-1a29-4789-989c-c6575121a284 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:36:04.187970 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da19df1-1a29-4789-989c-c6575121a284 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:36:04.188249 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da19df1-1a29-4789-989c-c6575121a284 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:36:04.188521 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da19df1-1a29-4789-989c-c6575121a284 None admin] PATH_INFO: `/v3/roles` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:36:04.189258 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da19df1-1a29-4789-989c-c6575121a284 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86827) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:36:04.189561 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da19df1-1a29-4789-989c-c6575121a284 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:36:04.191041 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da19df1-1a29-4789-989c-c6575121a284 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:36:04.200672 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 565/1102] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jul 27 10:36:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 10:36:04.206523 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd18d1d6-85fd-44f5-a220-7673009c37d3 None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:36:04.266584 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:36:04.319978 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd18d1d6-85fd-44f5-a220-7673009c37d3 None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:36:04.363123 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eea939d-a1d7-4aa3-bb29-6776dfc91bc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:36:04.363123 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eea939d-a1d7-4aa3-bb29-6776dfc91bc8 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:36:04.363123 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eea939d-a1d7-4aa3-bb29-6776dfc91bc8 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:36:04.363123 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eea939d-a1d7-4aa3-bb29-6776dfc91bc8 None admin] PATH_INFO: `/v3/users/cbf4568074bd483a8bcd1856c41f336d` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:36:04.364394 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eea939d-a1d7-4aa3-bb29-6776dfc91bc8 None admin] RBAC: Authorizing `identity:get_user(user_id=cbf4568074bd483a8bcd1856c41f336d)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:36:04.366392 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eea939d-a1d7-4aa3-bb29-6776dfc91bc8 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:36:04.372264 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 538/1103] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Jul 27 10:36:04 2022] GET /identity/v3/users/cbf4568074bd483a8bcd1856c41f336d => generated 285 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:36:04.373417 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da19df1-1a29-4789-989c-c6575121a284 None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:36:04.497704 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da19df1-1a29-4789-989c-c6575121a284 None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:36:04.539434 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf927c52-8b1a-4199-9caa-70702ee57e99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:36:04.540306 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf927c52-8b1a-4199-9caa-70702ee57e99 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:36:04.540734 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf927c52-8b1a-4199-9caa-70702ee57e99 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:36:04.541180 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf927c52-8b1a-4199-9caa-70702ee57e99 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:36:04.569769 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:36:04.631885 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf927c52-8b1a-4199-9caa-70702ee57e99 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:36:04.633266 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf927c52-8b1a-4199-9caa-70702ee57e99 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:36:04.731180 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 566/1104] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jul 27 10:36:04 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 27 10:36:04.737234 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eea939d-a1d7-4aa3-bb29-6776dfc91bc8 None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:36:04.836792 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eea939d-a1d7-4aa3-bb29-6776dfc91bc8 None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:36:04.859805 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:36:04.871332 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73108847-048b-4430-96e6-95fa6a5da62c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:36:04.871984 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73108847-048b-4430-96e6-95fa6a5da62c None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:36:04.872268 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73108847-048b-4430-96e6-95fa6a5da62c None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:36:04.872551 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73108847-048b-4430-96e6-95fa6a5da62c None admin] PATH_INFO: `/v3/projects/c8790f7d27cf4ee38f3ff22cdbe26e5f` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:36:04.873721 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73108847-048b-4430-96e6-95fa6a5da62c None admin] RBAC: Authorizing `identity:get_project(project_id=c8790f7d27cf4ee38f3ff22cdbe26e5f)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:36:04.875209 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73108847-048b-4430-96e6-95fa6a5da62c None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:36:04.876887 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 539/1105] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Jul 27 10:36:04 2022] GET /identity/v3/projects/c8790f7d27cf4ee38f3ff22cdbe26e5f => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:36:04.884207 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf927c52-8b1a-4199-9caa-70702ee57e99 None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:36:04.940230 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:36:04.987053 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf927c52-8b1a-4199-9caa-70702ee57e99 None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:36:05.024433 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c005949e-143f-4229-b484-525224cbd522 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:36:05.025056 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c005949e-143f-4229-b484-525224cbd522 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:36:05.025334 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c005949e-143f-4229-b484-525224cbd522 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:36:05.025606 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c005949e-143f-4229-b484-525224cbd522 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:36:05.026393 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c005949e-143f-4229-b484-525224cbd522 None admin] RBAC: Adding query filter params (role.id=21156560eb6b436bb567a40b4976f3f4, scope.project.id=c8790f7d27cf4ee38f3ff22cdbe26e5f, user.id=cbf4568074bd483a8bcd1856c41f336d) {{(pid=86827) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:36:05.026731 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c005949e-143f-4229-b484-525224cbd522 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:36:05.028158 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c005949e-143f-4229-b484-525224cbd522 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:36:05.034065 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 567/1106] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Jul 27 10:36:04 2022] GET /identity/v3/role_assignments?user.id=cbf4568074bd483a8bcd1856c41f336d&scope.project.id=c8790f7d27cf4ee38f3ff22cdbe26e5f&role.id=21156560eb6b436bb567a40b4976f3f4 => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:36:07.077325 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fe6f8f-67b2-4585-be39-fcdcf1bee62f None None] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:36:07.077839 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fe6f8f-67b2-4585-be39-fcdcf1bee62f None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:36:07.077951 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fe6f8f-67b2-4585-be39-fcdcf1bee62f None None] PATH_INFO: `/` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:36:07.079249 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 540/1107] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 10:36:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 10:36:07.084926 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc43e201-32a8-4df9-b3ef-adc8b337eaf8 None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:36:07.085342 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc43e201-32a8-4df9-b3ef-adc8b337eaf8 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:36:07.085485 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc43e201-32a8-4df9-b3ef-adc8b337eaf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:36:07.103781 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-dc43e201-32a8-4df9-b3ef-adc8b337eaf8 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:36:07.104962 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-dc43e201-32a8-4df9-b3ef-adc8b337eaf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:36:07.169008 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 568/1108] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 10:36:07 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 10:36:07.192989 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fe6f8f-67b2-4585-be39-fcdcf1bee62f None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:36:07.290404 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fe6f8f-67b2-4585-be39-fcdcf1bee62f None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:36:07.322447 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba1e082-401c-40bc-94b0-a51b5d139ddf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:36:07.323096 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba1e082-401c-40bc-94b0-a51b5d139ddf None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:36:07.323375 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba1e082-401c-40bc-94b0-a51b5d139ddf None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:36:07.323649 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba1e082-401c-40bc-94b0-a51b5d139ddf None admin] PATH_INFO: `/v3/domains/default` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:36:07.324773 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba1e082-401c-40bc-94b0-a51b5d139ddf None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:36:07.326198 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba1e082-401c-40bc-94b0-a51b5d139ddf None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:36:07.327776 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 541/1109] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 10:36:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:36:07.333467 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc43e201-32a8-4df9-b3ef-adc8b337eaf8 None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:36:07.381900 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:36:07.432171 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc43e201-32a8-4df9-b3ef-adc8b337eaf8 None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:36:07.464655 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64310bfa-d232-4ddf-afe7-c984c7120453 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:36:07.465207 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64310bfa-d232-4ddf-afe7-c984c7120453 None admin] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:36:07.465492 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64310bfa-d232-4ddf-afe7-c984c7120453 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:36:07.465768 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64310bfa-d232-4ddf-afe7-c984c7120453 None admin] PATH_INFO: `/v3/users` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:36:07.466454 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64310bfa-d232-4ddf-afe7-c984c7120453 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:36:07.467911 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64310bfa-d232-4ddf-afe7-c984c7120453 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:36:07.489079 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 569/1110] 10.222.0.36 () {60 vars in 1132 bytes} [Wed Jul 27 10:36:07 2022] POST /identity/v3/users => generated 286 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 10:36:09.526397 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf13665c-bfb2-4bc6-abdb-242bbb8f7452 None None] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:36:09.526964 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf13665c-bfb2-4bc6-abdb-242bbb8f7452 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:36:09.527197 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf13665c-bfb2-4bc6-abdb-242bbb8f7452 None None] PATH_INFO: `/` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:36:09.528567 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 542/1111] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 10:36:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 10:36:09.535566 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae655d4b-6b6b-4411-a7a5-951304a79f73 None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:36:09.535986 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae655d4b-6b6b-4411-a7a5-951304a79f73 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:36:09.536122 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae655d4b-6b6b-4411-a7a5-951304a79f73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:36:09.555738 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-ae655d4b-6b6b-4411-a7a5-951304a79f73 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:36:09.556928 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-ae655d4b-6b6b-4411-a7a5-951304a79f73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:36:09.622766 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 570/1112] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 10:36:09 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 10:36:09.645793 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf13665c-bfb2-4bc6-abdb-242bbb8f7452 None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:36:09.704258 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:36:09.745328 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf13665c-bfb2-4bc6-abdb-242bbb8f7452 None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:36:09.777072 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546dad54-811e-4268-94bc-d026f749fcc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:36:09.777637 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546dad54-811e-4268-94bc-d026f749fcc8 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:36:09.777919 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546dad54-811e-4268-94bc-d026f749fcc8 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:36:09.778191 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546dad54-811e-4268-94bc-d026f749fcc8 None admin] PATH_INFO: `/v3/roles/c3341f95596344309327efff99a20d5c` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:36:09.779592 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546dad54-811e-4268-94bc-d026f749fcc8 None admin] RBAC: Authorizing `identity:get_role(role_id=c3341f95596344309327efff99a20d5c)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:36:09.780951 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546dad54-811e-4268-94bc-d026f749fcc8 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:36:09.782263 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 543/1113] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Jul 27 10:36:09 2022] GET /identity/v3/roles/c3341f95596344309327efff99a20d5c => generated 223 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:36:09.788493 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae655d4b-6b6b-4411-a7a5-951304a79f73 None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:36:09.820949 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:36:09.892447 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae655d4b-6b6b-4411-a7a5-951304a79f73 None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:36:09.928511 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b140d0-008b-4ef6-8990-b1582c90de68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:36:09.929441 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b140d0-008b-4ef6-8990-b1582c90de68 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:36:09.929856 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b140d0-008b-4ef6-8990-b1582c90de68 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:36:09.930265 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b140d0-008b-4ef6-8990-b1582c90de68 None admin] PATH_INFO: `/v3/users/26974fa4d6e5448ba734d8d4d308bc79` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:36:09.948989 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b140d0-008b-4ef6-8990-b1582c90de68 None admin] RBAC: Authorizing `identity:get_user(user_id=26974fa4d6e5448ba734d8d4d308bc79)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:36:09.951302 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b140d0-008b-4ef6-8990-b1582c90de68 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:36:09.953891 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 571/1114] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Jul 27 10:36:09 2022] GET /identity/v3/users/26974fa4d6e5448ba734d8d4d308bc79 => generated 286 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:36:09.960118 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546dad54-811e-4268-94bc-d026f749fcc8 None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:36:10.057266 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546dad54-811e-4268-94bc-d026f749fcc8 None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:36:10.082145 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:36:10.090582 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc7f65a-fb0e-476c-82f8-0c184799e49b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:36:10.091220 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc7f65a-fb0e-476c-82f8-0c184799e49b None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:36:10.091500 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc7f65a-fb0e-476c-82f8-0c184799e49b None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:36:10.091772 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc7f65a-fb0e-476c-82f8-0c184799e49b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:36:10.155686 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc7f65a-fb0e-476c-82f8-0c184799e49b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:36:10.157101 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc7f65a-fb0e-476c-82f8-0c184799e49b None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:36:10.250697 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 544/1115] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jul 27 10:36:09 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 27 10:36:10.256629 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b140d0-008b-4ef6-8990-b1582c90de68 None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:36:10.345880 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:36:10.355557 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b140d0-008b-4ef6-8990-b1582c90de68 None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:36:10.389604 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c90da6-4ae0-4cc5-96f6-e2680fb0f214 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:36:10.390233 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c90da6-4ae0-4cc5-96f6-e2680fb0f214 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:36:10.390517 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c90da6-4ae0-4cc5-96f6-e2680fb0f214 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:36:10.390820 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c90da6-4ae0-4cc5-96f6-e2680fb0f214 None admin] PATH_INFO: `/v3/projects/c8790f7d27cf4ee38f3ff22cdbe26e5f` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:36:10.392055 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c90da6-4ae0-4cc5-96f6-e2680fb0f214 None admin] RBAC: Authorizing `identity:get_project(project_id=c8790f7d27cf4ee38f3ff22cdbe26e5f)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:36:10.393480 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c90da6-4ae0-4cc5-96f6-e2680fb0f214 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:36:10.395171 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 572/1116] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Jul 27 10:36:10 2022] GET /identity/v3/projects/c8790f7d27cf4ee38f3ff22cdbe26e5f => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:36:10.401733 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc7f65a-fb0e-476c-82f8-0c184799e49b None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:36:10.427917 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:36:10.496492 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc7f65a-fb0e-476c-82f8-0c184799e49b None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:36:10.529612 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466e6810-285b-49b4-928d-ba56f4f271b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:36:10.530217 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466e6810-285b-49b4-928d-ba56f4f271b8 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:36:10.530480 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466e6810-285b-49b4-928d-ba56f4f271b8 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:36:10.530779 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466e6810-285b-49b4-928d-ba56f4f271b8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:36:10.531564 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466e6810-285b-49b4-928d-ba56f4f271b8 None admin] RBAC: Adding query filter params (role.id=c3341f95596344309327efff99a20d5c, scope.project.id=c8790f7d27cf4ee38f3ff22cdbe26e5f, user.id=26974fa4d6e5448ba734d8d4d308bc79) {{(pid=86828) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:36:10.531869 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466e6810-285b-49b4-928d-ba56f4f271b8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:36:10.533284 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466e6810-285b-49b4-928d-ba56f4f271b8 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:36:10.538478 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 545/1117] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Jul 27 10:36:10 2022] GET /identity/v3/role_assignments?user.id=26974fa4d6e5448ba734d8d4d308bc79&scope.project.id=c8790f7d27cf4ee38f3ff22cdbe26e5f&role.id=c3341f95596344309327efff99a20d5c => generated 259 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:36:12.483692 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72cbec1-8eec-4101-af40-8313ab6e0564 None None] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:36:12.484323 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72cbec1-8eec-4101-af40-8313ab6e0564 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:36:12.484323 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72cbec1-8eec-4101-af40-8313ab6e0564 None None] PATH_INFO: `/` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:36:12.485593 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 573/1118] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 10:36:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 10:36:12.490674 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6106fe-36d1-449a-9154-1b96297efb57 None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:36:12.491114 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6106fe-36d1-449a-9154-1b96297efb57 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:36:12.491221 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6106fe-36d1-449a-9154-1b96297efb57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:36:12.509161 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-8b6106fe-36d1-449a-9154-1b96297efb57 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:36:12.510350 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-8b6106fe-36d1-449a-9154-1b96297efb57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:36:12.575354 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 546/1119] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 10:36:12 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 27 10:36:12.597543 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72cbec1-8eec-4101-af40-8313ab6e0564 None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:36:12.695713 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72cbec1-8eec-4101-af40-8313ab6e0564 None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:36:12.727880 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c0dbd6-ad48-4766-a919-824666110171 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:36:12.728503 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c0dbd6-ad48-4766-a919-824666110171 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:36:12.728782 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c0dbd6-ad48-4766-a919-824666110171 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:36:12.729056 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c0dbd6-ad48-4766-a919-824666110171 None admin] PATH_INFO: `/v3/roles/c3341f95596344309327efff99a20d5c` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:36:12.730459 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c0dbd6-ad48-4766-a919-824666110171 None admin] RBAC: Authorizing `identity:get_role(role_id=c3341f95596344309327efff99a20d5c)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:36:12.731838 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c0dbd6-ad48-4766-a919-824666110171 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:36:12.733146 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 574/1120] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Jul 27 10:36:12 2022] GET /identity/v3/roles/c3341f95596344309327efff99a20d5c => generated 223 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:36:12.738908 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6106fe-36d1-449a-9154-1b96297efb57 None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:36:12.762388 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:36:12.835028 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6106fe-36d1-449a-9154-1b96297efb57 None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:36:12.867201 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46ac7ac-b09b-4631-b206-e87478448cc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:36:12.867834 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46ac7ac-b09b-4631-b206-e87478448cc8 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:36:12.868120 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46ac7ac-b09b-4631-b206-e87478448cc8 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:36:12.868397 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46ac7ac-b09b-4631-b206-e87478448cc8 None admin] PATH_INFO: `/v3/users/26974fa4d6e5448ba734d8d4d308bc79` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:36:12.869550 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46ac7ac-b09b-4631-b206-e87478448cc8 None admin] RBAC: Authorizing `identity:get_user(user_id=26974fa4d6e5448ba734d8d4d308bc79)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:36:12.871018 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46ac7ac-b09b-4631-b206-e87478448cc8 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:36:12.872607 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 547/1121] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Jul 27 10:36:12 2022] GET /identity/v3/users/26974fa4d6e5448ba734d8d4d308bc79 => generated 286 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:36:12.878259 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c0dbd6-ad48-4766-a919-824666110171 None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:36:12.906979 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:36:12.977959 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c0dbd6-ad48-4766-a919-824666110171 None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:36:13.014498 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef856e3c-7ad8-45ed-b1fc-2fabc5561243 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:36:13.015085 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef856e3c-7ad8-45ed-b1fc-2fabc5561243 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:36:13.015373 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef856e3c-7ad8-45ed-b1fc-2fabc5561243 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:36:13.015646 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef856e3c-7ad8-45ed-b1fc-2fabc5561243 None admin] PATH_INFO: `/v3/projects/c8790f7d27cf4ee38f3ff22cdbe26e5f` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:36:13.016784 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef856e3c-7ad8-45ed-b1fc-2fabc5561243 None admin] RBAC: Authorizing `identity:get_project(project_id=c8790f7d27cf4ee38f3ff22cdbe26e5f)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:36:13.018207 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef856e3c-7ad8-45ed-b1fc-2fabc5561243 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:36:13.019897 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 575/1122] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Jul 27 10:36:12 2022] GET /identity/v3/projects/c8790f7d27cf4ee38f3ff22cdbe26e5f => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:36:13.025848 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46ac7ac-b09b-4631-b206-e87478448cc8 None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:36:13.112004 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:36:13.122306 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46ac7ac-b09b-4631-b206-e87478448cc8 None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:36:13.155489 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa439c4-439f-4917-9fe6-91e7dd4b04cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:36:13.156145 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa439c4-439f-4917-9fe6-91e7dd4b04cc None admin] REQUEST_METHOD: `PUT` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:36:13.156427 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa439c4-439f-4917-9fe6-91e7dd4b04cc None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:36:13.156702 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa439c4-439f-4917-9fe6-91e7dd4b04cc None admin] PATH_INFO: `/v3/projects/c8790f7d27cf4ee38f3ff22cdbe26e5f/users/26974fa4d6e5448ba734d8d4d308bc79/roles/c3341f95596344309327efff99a20d5c` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:36:13.158875 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa439c4-439f-4917-9fe6-91e7dd4b04cc None admin] RBAC: Authorizing `identity:create_grant(project_id=c8790f7d27cf4ee38f3ff22cdbe26e5f, user_id=26974fa4d6e5448ba734d8d4d308bc79, role_id=c3341f95596344309327efff99a20d5c)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:36:13.160289 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa439c4-439f-4917-9fe6-91e7dd4b04cc None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:36:13.172299 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 548/1123] 10.222.0.36 () {58 vars in 1439 bytes} [Wed Jul 27 10:36:13 2022] PUT /identity/v3/projects/c8790f7d27cf4ee38f3ff22cdbe26e5f/users/26974fa4d6e5448ba734d8d4d308bc79/roles/c3341f95596344309327efff99a20d5c => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:36:15.163898 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55ec7dc-3082-4765-9fae-29ceed64f660 None None] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:36:15.164452 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55ec7dc-3082-4765-9fae-29ceed64f660 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:36:15.164452 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55ec7dc-3082-4765-9fae-29ceed64f660 None None] PATH_INFO: `/` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:36:15.165800 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 576/1124] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 10:36:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 10:36:15.171021 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7597eb0d-2252-4503-9fd2-b6d6e705b675 None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:36:15.171414 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7597eb0d-2252-4503-9fd2-b6d6e705b675 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:36:15.171586 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7597eb0d-2252-4503-9fd2-b6d6e705b675 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:36:15.189813 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-7597eb0d-2252-4503-9fd2-b6d6e705b675 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:36:15.191037 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-7597eb0d-2252-4503-9fd2-b6d6e705b675 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:36:15.256265 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 549/1125] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 10:36:15 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 10:36:15.279897 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55ec7dc-3082-4765-9fae-29ceed64f660 None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:36:15.370434 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:36:15.377135 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55ec7dc-3082-4765-9fae-29ceed64f660 None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:36:15.410525 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7723c7-de61-4796-ba52-463474772a97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:36:15.411207 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7723c7-de61-4796-ba52-463474772a97 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:36:15.412069 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7723c7-de61-4796-ba52-463474772a97 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:36:15.412345 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7723c7-de61-4796-ba52-463474772a97 None admin] PATH_INFO: `/v3/roles/c3341f95596344309327efff99a20d5c` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:36:15.413756 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7723c7-de61-4796-ba52-463474772a97 None admin] RBAC: Authorizing `identity:get_role(role_id=c3341f95596344309327efff99a20d5c)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:36:15.415216 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7723c7-de61-4796-ba52-463474772a97 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:36:15.416535 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 577/1126] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Jul 27 10:36:15 2022] GET /identity/v3/roles/c3341f95596344309327efff99a20d5c => generated 223 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:36:15.422306 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7597eb0d-2252-4503-9fd2-b6d6e705b675 None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:36:15.499870 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:36:15.519702 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7597eb0d-2252-4503-9fd2-b6d6e705b675 None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:36:15.553009 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047606bd-d721-4c45-84e8-ebceaece16da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:36:15.553643 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047606bd-d721-4c45-84e8-ebceaece16da None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:36:15.553929 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047606bd-d721-4c45-84e8-ebceaece16da None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:36:15.554206 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047606bd-d721-4c45-84e8-ebceaece16da None admin] PATH_INFO: `/v3/users/26974fa4d6e5448ba734d8d4d308bc79` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:36:15.555391 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047606bd-d721-4c45-84e8-ebceaece16da None admin] RBAC: Authorizing `identity:get_user(user_id=26974fa4d6e5448ba734d8d4d308bc79)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:36:15.556821 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047606bd-d721-4c45-84e8-ebceaece16da None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:36:15.558397 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 550/1127] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Jul 27 10:36:15 2022] GET /identity/v3/users/26974fa4d6e5448ba734d8d4d308bc79 => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:36:15.564016 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7723c7-de61-4796-ba52-463474772a97 None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:36:15.660826 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7723c7-de61-4796-ba52-463474772a97 None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:36:15.693265 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25d008c-f515-4125-85ff-adc837cc7a2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:36:15.693866 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25d008c-f515-4125-85ff-adc837cc7a2e None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:36:15.694147 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25d008c-f515-4125-85ff-adc837cc7a2e None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:36:15.694423 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25d008c-f515-4125-85ff-adc837cc7a2e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:36:15.723716 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:36:15.760040 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25d008c-f515-4125-85ff-adc837cc7a2e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:36:15.761480 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25d008c-f515-4125-85ff-adc837cc7a2e None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:36:15.856147 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 578/1128] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jul 27 10:36:15 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 27 10:36:15.861929 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047606bd-d721-4c45-84e8-ebceaece16da None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:36:15.960729 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047606bd-d721-4c45-84e8-ebceaece16da None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:36:15.993490 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e5e897-599b-4ebb-bc48-be9cc2330aaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:36:15.994132 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e5e897-599b-4ebb-bc48-be9cc2330aaf None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:36:15.994416 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e5e897-599b-4ebb-bc48-be9cc2330aaf None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:36:15.994729 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e5e897-599b-4ebb-bc48-be9cc2330aaf None admin] PATH_INFO: `/v3/projects/c8790f7d27cf4ee38f3ff22cdbe26e5f` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:36:15.995878 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e5e897-599b-4ebb-bc48-be9cc2330aaf None admin] RBAC: Authorizing `identity:get_project(project_id=c8790f7d27cf4ee38f3ff22cdbe26e5f)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:36:15.997340 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e5e897-599b-4ebb-bc48-be9cc2330aaf None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:36:15.999031 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 551/1129] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Jul 27 10:36:15 2022] GET /identity/v3/projects/c8790f7d27cf4ee38f3ff22cdbe26e5f => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:36:16.005665 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25d008c-f515-4125-85ff-adc837cc7a2e None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:36:16.064857 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:36:16.101173 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25d008c-f515-4125-85ff-adc837cc7a2e None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:36:16.134409 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c77ba27-dd32-421b-8548-02fe7c3838c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:36:16.134975 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c77ba27-dd32-421b-8548-02fe7c3838c1 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:36:16.135258 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c77ba27-dd32-421b-8548-02fe7c3838c1 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:36:16.135529 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c77ba27-dd32-421b-8548-02fe7c3838c1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:36:16.136305 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c77ba27-dd32-421b-8548-02fe7c3838c1 None admin] RBAC: Adding query filter params (role.id=c3341f95596344309327efff99a20d5c, scope.project.id=c8790f7d27cf4ee38f3ff22cdbe26e5f, user.id=26974fa4d6e5448ba734d8d4d308bc79) {{(pid=86827) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:36:16.136609 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c77ba27-dd32-421b-8548-02fe7c3838c1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:36:16.138015 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c77ba27-dd32-421b-8548-02fe7c3838c1 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:36:16.143642 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 579/1130] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Jul 27 10:36:16 2022] GET /identity/v3/role_assignments?user.id=26974fa4d6e5448ba734d8d4d308bc79&scope.project.id=c8790f7d27cf4ee38f3ff22cdbe26e5f&role.id=c3341f95596344309327efff99a20d5c => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:36:18.145393 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60c3949-fb8f-450b-adb2-a31bc682ebf1 None None] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:36:18.146530 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60c3949-fb8f-450b-adb2-a31bc682ebf1 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:36:18.146530 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60c3949-fb8f-450b-adb2-a31bc682ebf1 None None] PATH_INFO: `/` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:36:18.147869 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 552/1131] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 10:36:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 10:36:18.152885 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b30d8da-a058-47c3-8af4-171ead80c08f None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:36:18.153328 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b30d8da-a058-47c3-8af4-171ead80c08f None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:36:18.153428 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b30d8da-a058-47c3-8af4-171ead80c08f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:36:18.171129 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-2b30d8da-a058-47c3-8af4-171ead80c08f None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:36:18.172305 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-2b30d8da-a058-47c3-8af4-171ead80c08f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:36:18.236311 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 580/1132] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 10:36:18 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 10:36:18.260119 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60c3949-fb8f-450b-adb2-a31bc682ebf1 None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:36:18.277565 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:36:18.357138 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60c3949-fb8f-450b-adb2-a31bc682ebf1 None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:36:18.390665 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08e743b-524c-4478-b265-9e4d217a8503 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:36:18.391288 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08e743b-524c-4478-b265-9e4d217a8503 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:36:18.391566 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08e743b-524c-4478-b265-9e4d217a8503 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:36:18.391839 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08e743b-524c-4478-b265-9e4d217a8503 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:36:18.392964 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08e743b-524c-4478-b265-9e4d217a8503 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:36:18.394382 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08e743b-524c-4478-b265-9e4d217a8503 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:36:18.395980 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 553/1133] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 10:36:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:36:18.401694 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b30d8da-a058-47c3-8af4-171ead80c08f None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:36:18.459421 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:36:18.500653 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b30d8da-a058-47c3-8af4-171ead80c08f None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:36:18.533872 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6a1d69-9877-4585-8446-4e8a01857e8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:36:18.534474 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6a1d69-9877-4585-8446-4e8a01857e8f None admin] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:36:18.534788 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6a1d69-9877-4585-8446-4e8a01857e8f None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:36:18.535067 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6a1d69-9877-4585-8446-4e8a01857e8f None admin] PATH_INFO: `/v3/projects` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:36:18.535756 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6a1d69-9877-4585-8446-4e8a01857e8f None admin] RBAC: Authorizing `identity:create_project()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:36:18.537798 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6a1d69-9877-4585-8446-4e8a01857e8f None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:36:18.564175 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 581/1134] 10.222.0.36 () {60 vars in 1141 bytes} [Wed Jul 27 10:36:18 2022] POST /identity/v3/projects => generated 311 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 10:36:20.609201 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cdcdb1-bfec-4b09-b84a-b0135727d4e3 None None] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:36:20.609749 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cdcdb1-bfec-4b09-b84a-b0135727d4e3 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:36:20.609749 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cdcdb1-bfec-4b09-b84a-b0135727d4e3 None None] PATH_INFO: `/` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:36:20.611112 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 554/1135] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 10:36:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 10:36:20.616737 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3de68d5-c1f8-4cb0-8abb-308b80dc6c28 None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:36:20.617124 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3de68d5-c1f8-4cb0-8abb-308b80dc6c28 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:36:20.617307 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3de68d5-c1f8-4cb0-8abb-308b80dc6c28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:36:20.636837 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-e3de68d5-c1f8-4cb0-8abb-308b80dc6c28 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:36:20.638025 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-e3de68d5-c1f8-4cb0-8abb-308b80dc6c28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:36:20.703186 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 582/1136] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 10:36:20 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 10:36:20.726774 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cdcdb1-bfec-4b09-b84a-b0135727d4e3 None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:36:20.808711 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:36:20.825610 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cdcdb1-bfec-4b09-b84a-b0135727d4e3 None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:36:20.858617 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2931135d-fd68-45f1-a9fb-b1702da4a7ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:36:20.859315 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2931135d-fd68-45f1-a9fb-b1702da4a7ac None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:36:20.859549 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2931135d-fd68-45f1-a9fb-b1702da4a7ac None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:36:20.859826 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2931135d-fd68-45f1-a9fb-b1702da4a7ac None admin] PATH_INFO: `/v3/domains/default` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:36:20.860965 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2931135d-fd68-45f1-a9fb-b1702da4a7ac None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:36:20.862391 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2931135d-fd68-45f1-a9fb-b1702da4a7ac None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:36:20.863983 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 555/1137] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 10:36:20 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:36:20.870265 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3de68d5-c1f8-4cb0-8abb-308b80dc6c28 None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:36:20.910476 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:36:20.969078 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3de68d5-c1f8-4cb0-8abb-308b80dc6c28 None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:36:21.002528 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e4f622-cf27-4aa7-b258-33aa70443f77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:36:21.003171 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e4f622-cf27-4aa7-b258-33aa70443f77 None admin] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:36:21.003456 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e4f622-cf27-4aa7-b258-33aa70443f77 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:36:21.003731 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e4f622-cf27-4aa7-b258-33aa70443f77 None admin] PATH_INFO: `/v3/users` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:36:21.004413 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e4f622-cf27-4aa7-b258-33aa70443f77 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:36:21.005875 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e4f622-cf27-4aa7-b258-33aa70443f77 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:36:21.026727 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 583/1138] 10.222.0.36 () {60 vars in 1132 bytes} [Wed Jul 27 10:36:20 2022] POST /identity/v3/users => generated 286 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 10:36:23.066071 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3bdd1e-6f57-45f3-bdaa-685d51b8d9a1 None None] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:36:23.066569 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3bdd1e-6f57-45f3-bdaa-685d51b8d9a1 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:36:23.066693 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3bdd1e-6f57-45f3-bdaa-685d51b8d9a1 None None] PATH_INFO: `/` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:36:23.068143 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 556/1139] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 10:36:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 10:36:23.073578 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1991be51-f8dc-4cfb-b27e-9bbd3b3e2cab None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:36:23.073982 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1991be51-f8dc-4cfb-b27e-9bbd3b3e2cab None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:36:23.074129 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1991be51-f8dc-4cfb-b27e-9bbd3b3e2cab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:36:23.092442 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-1991be51-f8dc-4cfb-b27e-9bbd3b3e2cab None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:36:23.093626 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-1991be51-f8dc-4cfb-b27e-9bbd3b3e2cab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:36:23.162104 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 584/1140] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 10:36:23 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 27 10:36:23.184874 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3bdd1e-6f57-45f3-bdaa-685d51b8d9a1 None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:36:23.283836 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3bdd1e-6f57-45f3-bdaa-685d51b8d9a1 None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:36:23.315680 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09defe3f-6931-4e97-9746-3bab2a89e78a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:36:23.316301 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09defe3f-6931-4e97-9746-3bab2a89e78a None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:36:23.316583 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09defe3f-6931-4e97-9746-3bab2a89e78a None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:36:23.316858 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09defe3f-6931-4e97-9746-3bab2a89e78a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:36:23.327991 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09defe3f-6931-4e97-9746-3bab2a89e78a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 10:36:23.333650 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application [None req-09defe3f-6931-4e97-9746-3bab2a89e78a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 10:36:23.333650 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:36:23.333650 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:36:23.333650 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:36:23.333650 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:36:23.333650 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:36:23.333650 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:36:23.333650 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:36:23.333650 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:36:23.333650 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:36:23.333650 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:36:23.333650 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:36:23.333650 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 10:36:23.333650 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 10:36:23.333650 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 10:36:23.333650 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 10:36:23.334851 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 10:36:23.334851 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 10:36:23.334851 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 10:36:23.334851 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 10:36:23.334851 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:36:23.334851 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:36:23.334851 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:36:23.334851 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:36:23.334851 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:36:23.334851 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:36:23.334851 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:36:23.334851 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application with Lock( Jul 27 10:36:23.334851 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:36:23.334851 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:36:23.334851 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:36:23.334851 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:36:23.335980 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:36:23.335980 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:36:23.335980 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:36:23.335980 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:36:23.335980 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 10:36:23.335980 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 10:36:23.335980 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 10:36:23.335980 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 10:36:23.335980 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 10:36:23.335980 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 10:36:23.335980 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 10:36:23.335980 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Jul 27 10:36:23.335980 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 557/1141] 10.222.0.36 () {56 vars in 1096 bytes} [Wed Jul 27 10:36:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 10:36:23.340747 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1991be51-f8dc-4cfb-b27e-9bbd3b3e2cab None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:36:23.366356 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:36:23.436551 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1991be51-f8dc-4cfb-b27e-9bbd3b3e2cab None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:36:23.469841 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56296628-a22b-44b7-a27c-0d73e9f80b47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:36:23.470446 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56296628-a22b-44b7-a27c-0d73e9f80b47 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:36:23.470753 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56296628-a22b-44b7-a27c-0d73e9f80b47 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:36:23.471027 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56296628-a22b-44b7-a27c-0d73e9f80b47 None admin] PATH_INFO: `/v3/roles` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:36:23.471731 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56296628-a22b-44b7-a27c-0d73e9f80b47 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86827) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:36:23.472032 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56296628-a22b-44b7-a27c-0d73e9f80b47 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:36:23.473381 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56296628-a22b-44b7-a27c-0d73e9f80b47 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:36:23.482089 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 585/1142] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jul 27 10:36:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:36:23.489802 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09defe3f-6931-4e97-9746-3bab2a89e78a None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:36:23.505476 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:36:23.587025 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09defe3f-6931-4e97-9746-3bab2a89e78a None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:36:23.619642 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b8ccd5-ffc6-49f2-a5eb-4c4aa5b962a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:36:23.620271 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b8ccd5-ffc6-49f2-a5eb-4c4aa5b962a1 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:36:23.620555 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b8ccd5-ffc6-49f2-a5eb-4c4aa5b962a1 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:36:23.620833 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b8ccd5-ffc6-49f2-a5eb-4c4aa5b962a1 None admin] PATH_INFO: `/v3/users/48d861dacdac4d438c96af621b16e875` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:36:23.634290 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b8ccd5-ffc6-49f2-a5eb-4c4aa5b962a1 None admin] RBAC: Authorizing `identity:get_user(user_id=48d861dacdac4d438c96af621b16e875)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:36:23.635759 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b8ccd5-ffc6-49f2-a5eb-4c4aa5b962a1 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:36:23.637362 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 558/1143] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Jul 27 10:36:23 2022] GET /identity/v3/users/48d861dacdac4d438c96af621b16e875 => generated 286 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:36:23.643089 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56296628-a22b-44b7-a27c-0d73e9f80b47 None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:36:23.727678 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:36:23.740378 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56296628-a22b-44b7-a27c-0d73e9f80b47 None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:36:23.776759 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9dfe73f-8996-40fa-84be-a50084423fed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:36:23.777384 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dfe73f-8996-40fa-84be-a50084423fed None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:36:23.777666 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dfe73f-8996-40fa-84be-a50084423fed None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:36:23.777939 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dfe73f-8996-40fa-84be-a50084423fed None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:36:23.865276 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9dfe73f-8996-40fa-84be-a50084423fed None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:36:23.866666 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9dfe73f-8996-40fa-84be-a50084423fed None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:36:23.950877 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:36:23.963476 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 586/1144] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jul 27 10:36:23 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 27 10:36:23.969360 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b8ccd5-ffc6-49f2-a5eb-4c4aa5b962a1 None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:36:24.037582 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:36:24.067998 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b8ccd5-ffc6-49f2-a5eb-4c4aa5b962a1 None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:36:24.113810 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a3f96d-53b3-4d5f-8c20-f4c93c12507c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:36:24.114460 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a3f96d-53b3-4d5f-8c20-f4c93c12507c None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:36:24.114780 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a3f96d-53b3-4d5f-8c20-f4c93c12507c None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:36:24.115048 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a3f96d-53b3-4d5f-8c20-f4c93c12507c None admin] PATH_INFO: `/v3/projects/01567e353e804bdc86977c25af6c8336` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:36:24.121238 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a3f96d-53b3-4d5f-8c20-f4c93c12507c None admin] RBAC: Authorizing `identity:get_project(project_id=01567e353e804bdc86977c25af6c8336)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:36:24.122737 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a3f96d-53b3-4d5f-8c20-f4c93c12507c None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:36:24.124440 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 559/1145] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Jul 27 10:36:23 2022] GET /identity/v3/projects/01567e353e804bdc86977c25af6c8336 => generated 311 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:36:24.130733 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9dfe73f-8996-40fa-84be-a50084423fed None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:36:24.231085 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9dfe73f-8996-40fa-84be-a50084423fed None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:36:24.264233 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec17e524-8e64-4b99-bccf-aaeb7b9df977 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:36:24.264842 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec17e524-8e64-4b99-bccf-aaeb7b9df977 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:36:24.265125 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec17e524-8e64-4b99-bccf-aaeb7b9df977 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:36:24.265399 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec17e524-8e64-4b99-bccf-aaeb7b9df977 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:36:24.266188 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec17e524-8e64-4b99-bccf-aaeb7b9df977 None admin] RBAC: Adding query filter params (role.id=21156560eb6b436bb567a40b4976f3f4, scope.project.id=01567e353e804bdc86977c25af6c8336, user.id=48d861dacdac4d438c96af621b16e875) {{(pid=86827) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:36:24.266500 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec17e524-8e64-4b99-bccf-aaeb7b9df977 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:36:24.267959 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec17e524-8e64-4b99-bccf-aaeb7b9df977 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:36:24.273832 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 587/1146] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Jul 27 10:36:24 2022] GET /identity/v3/role_assignments?user.id=48d861dacdac4d438c96af621b16e875&scope.project.id=01567e353e804bdc86977c25af6c8336&role.id=21156560eb6b436bb567a40b4976f3f4 => generated 259 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:36:26.241589 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84a16a4-1c35-4e52-9c1b-87e4b3ff2185 None None] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:36:26.242097 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84a16a4-1c35-4e52-9c1b-87e4b3ff2185 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:36:26.242097 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84a16a4-1c35-4e52-9c1b-87e4b3ff2185 None None] PATH_INFO: `/` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:36:26.243468 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 560/1147] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 10:36:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 10:36:26.248509 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f567d6-711d-42b0-a907-12c48982ea1f None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:36:26.248874 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f567d6-711d-42b0-a907-12c48982ea1f None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:36:26.249060 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f567d6-711d-42b0-a907-12c48982ea1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:36:26.267343 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-49f567d6-711d-42b0-a907-12c48982ea1f None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:36:26.268501 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-49f567d6-711d-42b0-a907-12c48982ea1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:36:26.313011 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:36:26.334022 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 588/1148] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 10:36:26 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 10:36:26.355321 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84a16a4-1c35-4e52-9c1b-87e4b3ff2185 None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:36:26.453553 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84a16a4-1c35-4e52-9c1b-87e4b3ff2185 None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:36:26.487371 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6465ab5f-fb9a-4a75-97c2-85450de3671c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:36:26.487999 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6465ab5f-fb9a-4a75-97c2-85450de3671c None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:36:26.488283 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6465ab5f-fb9a-4a75-97c2-85450de3671c None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:36:26.488559 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6465ab5f-fb9a-4a75-97c2-85450de3671c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:36:26.495735 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:36:26.499820 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6465ab5f-fb9a-4a75-97c2-85450de3671c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 10:36:26.505602 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application [None req-6465ab5f-fb9a-4a75-97c2-85450de3671c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 10:36:26.505602 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:36:26.505602 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:36:26.505602 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:36:26.505602 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:36:26.505602 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:36:26.505602 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:36:26.505602 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:36:26.505602 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:36:26.505602 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:36:26.505602 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:36:26.505602 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:36:26.505602 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 10:36:26.505602 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 10:36:26.505602 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 10:36:26.505602 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 10:36:26.506762 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 10:36:26.506762 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 10:36:26.506762 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 10:36:26.506762 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 10:36:26.506762 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:36:26.506762 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:36:26.506762 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:36:26.506762 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:36:26.506762 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:36:26.506762 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:36:26.506762 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:36:26.506762 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application with Lock( Jul 27 10:36:26.506762 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:36:26.506762 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:36:26.506762 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:36:26.506762 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:36:26.507847 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:36:26.507847 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:36:26.507847 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:36:26.507847 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:36:26.507847 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 10:36:26.507847 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 10:36:26.507847 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 10:36:26.507847 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 10:36:26.507847 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 10:36:26.507847 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 10:36:26.507847 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 10:36:26.507847 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Jul 27 10:36:26.507847 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 561/1149] 10.222.0.36 () {56 vars in 1096 bytes} [Wed Jul 27 10:36:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 10:36:26.512952 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f567d6-711d-42b0-a907-12c48982ea1f None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:36:26.606700 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f567d6-711d-42b0-a907-12c48982ea1f None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:36:26.640403 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f3e0b4-08b1-4aa6-b673-1160ac615d56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:36:26.641013 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f3e0b4-08b1-4aa6-b673-1160ac615d56 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:36:26.641294 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f3e0b4-08b1-4aa6-b673-1160ac615d56 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:36:26.641570 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f3e0b4-08b1-4aa6-b673-1160ac615d56 None admin] PATH_INFO: `/v3/roles` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:36:26.642279 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f3e0b4-08b1-4aa6-b673-1160ac615d56 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86827) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:36:26.642582 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f3e0b4-08b1-4aa6-b673-1160ac615d56 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:36:26.643975 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f3e0b4-08b1-4aa6-b673-1160ac615d56 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:36:26.651760 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 589/1150] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jul 27 10:36:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:36:26.657286 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6465ab5f-fb9a-4a75-97c2-85450de3671c None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:36:26.753045 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6465ab5f-fb9a-4a75-97c2-85450de3671c None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:36:26.785253 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bc3a80-54b6-4a3e-9fc6-c344ed83d79d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:36:26.785895 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bc3a80-54b6-4a3e-9fc6-c344ed83d79d None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:36:26.786174 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bc3a80-54b6-4a3e-9fc6-c344ed83d79d None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:36:26.786443 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bc3a80-54b6-4a3e-9fc6-c344ed83d79d None admin] PATH_INFO: `/v3/users/48d861dacdac4d438c96af621b16e875` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:36:26.787606 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3bc3a80-54b6-4a3e-9fc6-c344ed83d79d None admin] RBAC: Authorizing `identity:get_user(user_id=48d861dacdac4d438c96af621b16e875)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:36:26.789020 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3bc3a80-54b6-4a3e-9fc6-c344ed83d79d None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:36:26.790573 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 562/1151] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Jul 27 10:36:26 2022] GET /identity/v3/users/48d861dacdac4d438c96af621b16e875 => generated 286 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:36:26.796186 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f3e0b4-08b1-4aa6-b673-1160ac615d56 None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:36:26.841299 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:36:26.894621 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f3e0b4-08b1-4aa6-b673-1160ac615d56 None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:36:26.927408 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6332bab1-5931-4ff6-be8f-dfd45d957a90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:36:26.928037 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6332bab1-5931-4ff6-be8f-dfd45d957a90 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:36:26.928320 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6332bab1-5931-4ff6-be8f-dfd45d957a90 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:36:26.928598 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6332bab1-5931-4ff6-be8f-dfd45d957a90 None admin] PATH_INFO: `/v3/projects/01567e353e804bdc86977c25af6c8336` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:36:26.929744 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6332bab1-5931-4ff6-be8f-dfd45d957a90 None admin] RBAC: Authorizing `identity:get_project(project_id=01567e353e804bdc86977c25af6c8336)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:36:26.931207 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6332bab1-5931-4ff6-be8f-dfd45d957a90 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:36:26.932860 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 590/1152] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Jul 27 10:36:26 2022] GET /identity/v3/projects/01567e353e804bdc86977c25af6c8336 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:36:26.938841 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bc3a80-54b6-4a3e-9fc6-c344ed83d79d None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:36:27.002381 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:36:27.036992 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bc3a80-54b6-4a3e-9fc6-c344ed83d79d None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:36:27.069154 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41786dc5-45f9-4852-8c59-25fc4a10dd60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:36:27.069810 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41786dc5-45f9-4852-8c59-25fc4a10dd60 None admin] REQUEST_METHOD: `PUT` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:36:27.070091 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41786dc5-45f9-4852-8c59-25fc4a10dd60 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:36:27.070366 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41786dc5-45f9-4852-8c59-25fc4a10dd60 None admin] PATH_INFO: `/v3/projects/01567e353e804bdc86977c25af6c8336/users/48d861dacdac4d438c96af621b16e875/roles/21156560eb6b436bb567a40b4976f3f4` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:36:27.072302 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41786dc5-45f9-4852-8c59-25fc4a10dd60 None admin] RBAC: Authorizing `identity:create_grant(project_id=01567e353e804bdc86977c25af6c8336, user_id=48d861dacdac4d438c96af621b16e875, role_id=21156560eb6b436bb567a40b4976f3f4)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:36:27.073726 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41786dc5-45f9-4852-8c59-25fc4a10dd60 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:36:27.086382 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 563/1153] 10.222.0.36 () {58 vars in 1439 bytes} [Wed Jul 27 10:36:26 2022] PUT /identity/v3/projects/01567e353e804bdc86977c25af6c8336/users/48d861dacdac4d438c96af621b16e875/roles/21156560eb6b436bb567a40b4976f3f4 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:36:29.043489 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4e91ff-c8dc-40a8-9bd8-22a74e807334 None None] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:36:29.043996 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4e91ff-c8dc-40a8-9bd8-22a74e807334 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:36:29.044088 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4e91ff-c8dc-40a8-9bd8-22a74e807334 None None] PATH_INFO: `/` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:36:29.045388 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 591/1154] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 10:36:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 10:36:29.050755 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d2eaa5-8e8d-49a5-9fef-c38f1a2300d5 None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:36:29.051166 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d2eaa5-8e8d-49a5-9fef-c38f1a2300d5 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:36:29.051340 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d2eaa5-8e8d-49a5-9fef-c38f1a2300d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:36:29.070205 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-d8d2eaa5-8e8d-49a5-9fef-c38f1a2300d5 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:36:29.071421 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-d8d2eaa5-8e8d-49a5-9fef-c38f1a2300d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:36:29.136990 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 564/1155] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 10:36:29 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 10:36:29.159758 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4e91ff-c8dc-40a8-9bd8-22a74e807334 None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:36:29.265518 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4e91ff-c8dc-40a8-9bd8-22a74e807334 None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:36:29.277710 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:36:29.298701 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf2efb6-dea4-4121-81c7-0cd97ab38d4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:36:29.299339 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf2efb6-dea4-4121-81c7-0cd97ab38d4f None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:36:29.299620 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf2efb6-dea4-4121-81c7-0cd97ab38d4f None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:36:29.299892 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf2efb6-dea4-4121-81c7-0cd97ab38d4f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:36:29.310617 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf2efb6-dea4-4121-81c7-0cd97ab38d4f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 10:36:29.315520 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application [None req-3bf2efb6-dea4-4121-81c7-0cd97ab38d4f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 10:36:29.315520 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:36:29.315520 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:36:29.315520 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:36:29.315520 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:36:29.315520 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:36:29.315520 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:36:29.315520 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:36:29.315520 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:36:29.315520 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:36:29.315520 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:36:29.315520 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:36:29.315520 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 10:36:29.315520 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 10:36:29.315520 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 10:36:29.315520 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 10:36:29.316619 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 10:36:29.316619 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 10:36:29.316619 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 10:36:29.316619 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 10:36:29.316619 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:36:29.316619 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:36:29.316619 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:36:29.316619 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:36:29.316619 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:36:29.316619 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:36:29.316619 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:36:29.316619 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application with Lock( Jul 27 10:36:29.316619 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:36:29.316619 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:36:29.316619 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:36:29.316619 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:36:29.317712 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:36:29.317712 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:36:29.317712 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:36:29.317712 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:36:29.317712 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 10:36:29.317712 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 10:36:29.317712 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 10:36:29.317712 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 10:36:29.317712 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 10:36:29.317712 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 10:36:29.317712 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 10:36:29.317712 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Jul 27 10:36:29.317712 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 592/1156] 10.222.0.36 () {56 vars in 1096 bytes} [Wed Jul 27 10:36:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 10:36:29.322913 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d2eaa5-8e8d-49a5-9fef-c38f1a2300d5 None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:36:29.377248 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:36:29.419391 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d2eaa5-8e8d-49a5-9fef-c38f1a2300d5 None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:36:29.451590 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75644d39-c6da-498e-812d-bc7fce51ff1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:36:29.452194 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75644d39-c6da-498e-812d-bc7fce51ff1c None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:36:29.452472 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75644d39-c6da-498e-812d-bc7fce51ff1c None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:36:29.452746 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75644d39-c6da-498e-812d-bc7fce51ff1c None admin] PATH_INFO: `/v3/roles` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:36:29.453452 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75644d39-c6da-498e-812d-bc7fce51ff1c None admin] RBAC: Adding query filter params (name=admin) {{(pid=86828) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:36:29.453750 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75644d39-c6da-498e-812d-bc7fce51ff1c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:36:29.455150 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75644d39-c6da-498e-812d-bc7fce51ff1c None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:36:29.463376 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 565/1157] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jul 27 10:36:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:36:29.469156 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf2efb6-dea4-4121-81c7-0cd97ab38d4f None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:36:29.568022 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf2efb6-dea4-4121-81c7-0cd97ab38d4f None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:36:29.601559 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdda605-76a5-4d2a-8586-ca3535276b2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:36:29.602185 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdda605-76a5-4d2a-8586-ca3535276b2c None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:36:29.602464 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdda605-76a5-4d2a-8586-ca3535276b2c None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:36:29.602766 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdda605-76a5-4d2a-8586-ca3535276b2c None admin] PATH_INFO: `/v3/users/48d861dacdac4d438c96af621b16e875` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:36:29.603930 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bdda605-76a5-4d2a-8586-ca3535276b2c None admin] RBAC: Authorizing `identity:get_user(user_id=48d861dacdac4d438c96af621b16e875)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:36:29.605346 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bdda605-76a5-4d2a-8586-ca3535276b2c None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:36:29.606965 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 593/1158] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Jul 27 10:36:29 2022] GET /identity/v3/users/48d861dacdac4d438c96af621b16e875 => generated 286 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:36:29.612674 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75644d39-c6da-498e-812d-bc7fce51ff1c None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:36:29.708824 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75644d39-c6da-498e-812d-bc7fce51ff1c None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:36:29.725381 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:36:29.741734 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a7f91d-68a6-4d1b-902a-ce0205650a1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:36:29.742354 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a7f91d-68a6-4d1b-902a-ce0205650a1f None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:36:29.742660 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a7f91d-68a6-4d1b-902a-ce0205650a1f None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:36:29.742938 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a7f91d-68a6-4d1b-902a-ce0205650a1f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:36:29.807227 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a7f91d-68a6-4d1b-902a-ce0205650a1f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:36:29.808678 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a7f91d-68a6-4d1b-902a-ce0205650a1f None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:36:29.903979 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 566/1159] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jul 27 10:36:29 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 27 10:36:29.909730 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdda605-76a5-4d2a-8586-ca3535276b2c None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:36:29.955971 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:36:30.014267 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdda605-76a5-4d2a-8586-ca3535276b2c None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:36:30.046980 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0751d66e-3e31-4453-8ae0-53372172adec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:36:30.047615 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0751d66e-3e31-4453-8ae0-53372172adec None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:36:30.047899 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0751d66e-3e31-4453-8ae0-53372172adec None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:36:30.048176 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0751d66e-3e31-4453-8ae0-53372172adec None admin] PATH_INFO: `/v3/projects/01567e353e804bdc86977c25af6c8336` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:36:30.049329 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0751d66e-3e31-4453-8ae0-53372172adec None admin] RBAC: Authorizing `identity:get_project(project_id=01567e353e804bdc86977c25af6c8336)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:36:30.050790 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0751d66e-3e31-4453-8ae0-53372172adec None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:36:30.052786 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 594/1160] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Jul 27 10:36:29 2022] GET /identity/v3/projects/01567e353e804bdc86977c25af6c8336 => generated 311 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:36:30.058984 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a7f91d-68a6-4d1b-902a-ce0205650a1f None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:36:30.077321 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:36:30.156854 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a7f91d-68a6-4d1b-902a-ce0205650a1f None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:36:30.189026 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30ab982-b853-4754-bee2-23559d07d4b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:36:30.189635 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30ab982-b853-4754-bee2-23559d07d4b6 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:36:30.189920 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30ab982-b853-4754-bee2-23559d07d4b6 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:36:30.190197 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30ab982-b853-4754-bee2-23559d07d4b6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:36:30.191015 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30ab982-b853-4754-bee2-23559d07d4b6 None admin] RBAC: Adding query filter params (role.id=21156560eb6b436bb567a40b4976f3f4, scope.project.id=01567e353e804bdc86977c25af6c8336, user.id=48d861dacdac4d438c96af621b16e875) {{(pid=86828) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:36:30.191328 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30ab982-b853-4754-bee2-23559d07d4b6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:36:30.193276 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30ab982-b853-4754-bee2-23559d07d4b6 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:36:30.198943 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 567/1161] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Jul 27 10:36:30 2022] GET /identity/v3/role_assignments?user.id=48d861dacdac4d438c96af621b16e875&scope.project.id=01567e353e804bdc86977c25af6c8336&role.id=21156560eb6b436bb567a40b4976f3f4 => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 10:36:32.201753 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd91b29-3944-4b96-8b9a-76e796a6174e None None] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:36:32.202269 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd91b29-3944-4b96-8b9a-76e796a6174e None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:36:32.202269 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd91b29-3944-4b96-8b9a-76e796a6174e None None] PATH_INFO: `/` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:36:32.203648 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 595/1162] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 10:36:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 10:36:32.208707 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c638ca04-c558-43ec-8f21-aa9b82cd85ce None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:36:32.209127 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c638ca04-c558-43ec-8f21-aa9b82cd85ce None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:36:32.209263 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c638ca04-c558-43ec-8f21-aa9b82cd85ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:36:32.227192 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-c638ca04-c558-43ec-8f21-aa9b82cd85ce None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:36:32.228390 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-c638ca04-c558-43ec-8f21-aa9b82cd85ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:36:32.299354 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 568/1163] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 10:36:32 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 27 10:36:32.322200 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd91b29-3944-4b96-8b9a-76e796a6174e None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:36:32.420989 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd91b29-3944-4b96-8b9a-76e796a6174e None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:36:32.429969 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:36:32.454899 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7137408-6825-4cf3-abac-e9ab156ccc5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:36:32.455444 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7137408-6825-4cf3-abac-e9ab156ccc5e None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:36:32.455730 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7137408-6825-4cf3-abac-e9ab156ccc5e None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:36:32.456002 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7137408-6825-4cf3-abac-e9ab156ccc5e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:36:32.521496 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7137408-6825-4cf3-abac-e9ab156ccc5e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:36:32.522970 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7137408-6825-4cf3-abac-e9ab156ccc5e None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:36:32.635189 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 596/1164] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jul 27 10:36:32 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 27 10:36:32.641523 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c638ca04-c558-43ec-8f21-aa9b82cd85ce None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:36:32.655239 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:36:32.738294 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c638ca04-c558-43ec-8f21-aa9b82cd85ce None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:36:32.771849 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb180cc9-4492-41c2-9dd5-b80b8cab0391 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:36:32.772491 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb180cc9-4492-41c2-9dd5-b80b8cab0391 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:36:32.772774 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb180cc9-4492-41c2-9dd5-b80b8cab0391 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:36:32.773051 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb180cc9-4492-41c2-9dd5-b80b8cab0391 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:36:32.779563 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb180cc9-4492-41c2-9dd5-b80b8cab0391 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:36:32.781001 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb180cc9-4492-41c2-9dd5-b80b8cab0391 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:36:32.792818 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application [None req-cb180cc9-4492-41c2-9dd5-b80b8cab0391 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 27 10:36:32.792818 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:36:32.792818 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jul 27 10:36:32.792818 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jul 27 10:36:32.792818 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 10:36:32.792818 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 10:36:32.792818 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 10:36:32.792818 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 10:36:32.792818 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jul 27 10:36:32.792818 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Jul 27 10:36:32.792818 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 10:36:32.792818 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Jul 27 10:36:32.792818 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:36:32.792818 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:36:32.792818 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:36:32.792818 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:36:32.792818 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:36:32.794019 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:36:32.794019 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:36:32.794019 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:36:32.794019 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:36:32.794019 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:36:32.794019 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:36:32.794019 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 10:36:32.794019 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 10:36:32.794019 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 10:36:32.794019 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 10:36:32.794019 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:36:32.794019 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:36:32.794019 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:36:32.794019 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:36:32.794019 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:36:32.794019 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:36:32.795121 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:36:32.795121 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application with Lock( Jul 27 10:36:32.795121 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:36:32.795121 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:36:32.795121 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:36:32.795121 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:36:32.795121 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:36:32.795121 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:36:32.795121 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:36:32.795121 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:36:32.795121 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jul 27 10:36:32.795121 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 10:36:32.795121 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 27 10:36:32.795121 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Jul 27 10:36:32.795121 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 569/1165] 10.222.0.36 () {56 vars in 1117 bytes} [Wed Jul 27 10:36:32 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 10:36:32.799943 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7137408-6825-4cf3-abac-e9ab156ccc5e None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:36:32.810557 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:36:32.897167 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7137408-6825-4cf3-abac-e9ab156ccc5e None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:36:32.929901 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1324821-4601-4bc3-8e94-bbc6e3b63643 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:36:32.930504 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1324821-4601-4bc3-8e94-bbc6e3b63643 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:36:32.930814 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1324821-4601-4bc3-8e94-bbc6e3b63643 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:36:32.931092 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1324821-4601-4bc3-8e94-bbc6e3b63643 None admin] PATH_INFO: `/v3/domains` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:36:32.931778 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1324821-4601-4bc3-8e94-bbc6e3b63643 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=86827) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:36:32.932084 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1324821-4601-4bc3-8e94-bbc6e3b63643 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:36:32.933443 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1324821-4601-4bc3-8e94-bbc6e3b63643 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:36:32.940424 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 597/1166] 10.222.0.36 () {56 vars in 1115 bytes} [Wed Jul 27 10:36:32 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:36:34.908730 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b09a02d-85b0-4099-8f46-5055db51f96a None None] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:36:34.909253 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b09a02d-85b0-4099-8f46-5055db51f96a None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:36:34.909365 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b09a02d-85b0-4099-8f46-5055db51f96a None None] PATH_INFO: `/` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:36:34.910679 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 570/1167] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 10:36:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 10:36:34.915858 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c5beb1-7a70-48ad-9c68-8fcb446f16d8 None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:36:34.916252 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c5beb1-7a70-48ad-9c68-8fcb446f16d8 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:36:34.916420 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c5beb1-7a70-48ad-9c68-8fcb446f16d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:36:34.934022 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-b1c5beb1-7a70-48ad-9c68-8fcb446f16d8 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:36:34.935229 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-b1c5beb1-7a70-48ad-9c68-8fcb446f16d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:36:35.000414 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 598/1168] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 10:36:34 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 10:36:35.023766 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b09a02d-85b0-4099-8f46-5055db51f96a None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:36:35.093066 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:36:35.124508 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b09a02d-85b0-4099-8f46-5055db51f96a None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:36:35.157704 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26250a3-780b-4037-9fd0-02f1bc17e0be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:36:35.158328 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26250a3-780b-4037-9fd0-02f1bc17e0be None admin] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:36:35.158635 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26250a3-780b-4037-9fd0-02f1bc17e0be None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:36:35.158956 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26250a3-780b-4037-9fd0-02f1bc17e0be None admin] PATH_INFO: `/v3/domains` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:36:35.159627 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26250a3-780b-4037-9fd0-02f1bc17e0be None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:36:35.161578 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26250a3-780b-4037-9fd0-02f1bc17e0be None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:36:35.177136 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 571/1169] 10.222.0.36 () {60 vars in 1138 bytes} [Wed Jul 27 10:36:35 2022] POST /identity/v3/domains => generated 267 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 10:36:37.197154 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2efb09-4c17-4ead-ad07-4bc0bd2ae1c7 None None] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:36:37.197667 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2efb09-4c17-4ead-ad07-4bc0bd2ae1c7 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:36:37.197667 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2efb09-4c17-4ead-ad07-4bc0bd2ae1c7 None None] PATH_INFO: `/` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:36:37.199037 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 599/1170] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 10:36:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 10:36:37.204127 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39febf8-5cbd-4322-b850-3438c03febf4 None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:36:37.204500 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39febf8-5cbd-4322-b850-3438c03febf4 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:36:37.204693 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39febf8-5cbd-4322-b850-3438c03febf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:36:37.223022 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-f39febf8-5cbd-4322-b850-3438c03febf4 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:36:37.224223 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-f39febf8-5cbd-4322-b850-3438c03febf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:36:37.289428 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 572/1171] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 10:36:37 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 10:36:37.311748 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2efb09-4c17-4ead-ad07-4bc0bd2ae1c7 None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:36:37.314585 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:36:37.418375 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2efb09-4c17-4ead-ad07-4bc0bd2ae1c7 None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:36:37.453372 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4868ab75-ae74-460d-ad8d-f5e8768c5a4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:36:37.453935 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4868ab75-ae74-460d-ad8d-f5e8768c5a4e None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:36:37.454220 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4868ab75-ae74-460d-ad8d-f5e8768c5a4e None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:36:37.454500 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4868ab75-ae74-460d-ad8d-f5e8768c5a4e None admin] PATH_INFO: `/v3/domains/a8b6e383e1c945608bfb70d4a9adb5b2` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:36:37.464770 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4868ab75-ae74-460d-ad8d-f5e8768c5a4e None admin] RBAC: Authorizing `identity:get_domain(domain_id=a8b6e383e1c945608bfb70d4a9adb5b2)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:36:37.466219 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4868ab75-ae74-460d-ad8d-f5e8768c5a4e None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:36:37.467912 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 600/1172] 10.222.0.36 () {56 vars in 1183 bytes} [Wed Jul 27 10:36:37 2022] GET /identity/v3/domains/a8b6e383e1c945608bfb70d4a9adb5b2 => generated 267 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:36:37.473896 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39febf8-5cbd-4322-b850-3438c03febf4 None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:36:37.532648 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:36:37.570438 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39febf8-5cbd-4322-b850-3438c03febf4 None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:36:37.602249 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c17516-58c1-4f2c-8653-ae0db7485e9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:36:37.602899 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c17516-58c1-4f2c-8653-ae0db7485e9f None admin] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:36:37.603182 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c17516-58c1-4f2c-8653-ae0db7485e9f None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:36:37.603458 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c17516-58c1-4f2c-8653-ae0db7485e9f None admin] PATH_INFO: `/v3/projects` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:36:37.604140 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c17516-58c1-4f2c-8653-ae0db7485e9f None admin] RBAC: Authorizing `identity:create_project()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:36:37.605557 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c17516-58c1-4f2c-8653-ae0db7485e9f None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:36:37.632237 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 573/1173] 10.222.0.36 () {60 vars in 1141 bytes} [Wed Jul 27 10:36:37 2022] POST /identity/v3/projects => generated 361 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 10:36:39.669182 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4643872d-feb6-45e2-ae33-37966fe88534 None None] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:36:39.669740 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4643872d-feb6-45e2-ae33-37966fe88534 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:36:39.669740 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4643872d-feb6-45e2-ae33-37966fe88534 None None] PATH_INFO: `/` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:36:39.671153 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 601/1174] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 10:36:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 10:36:39.676780 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bae478-5bc5-44a2-8e0d-37e6b69db151 None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:36:39.677216 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bae478-5bc5-44a2-8e0d-37e6b69db151 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:36:39.677321 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bae478-5bc5-44a2-8e0d-37e6b69db151 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:36:39.695205 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-c2bae478-5bc5-44a2-8e0d-37e6b69db151 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:36:39.696396 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-c2bae478-5bc5-44a2-8e0d-37e6b69db151 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:36:39.761432 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 574/1175] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 10:36:39 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 10:36:39.785766 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4643872d-feb6-45e2-ae33-37966fe88534 None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:36:39.847354 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:36:39.886791 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4643872d-feb6-45e2-ae33-37966fe88534 None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:36:39.920301 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4067b5fb-692d-4368-bd00-497c212183b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:36:39.920865 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4067b5fb-692d-4368-bd00-497c212183b9 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:36:39.921147 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4067b5fb-692d-4368-bd00-497c212183b9 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:36:39.921424 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4067b5fb-692d-4368-bd00-497c212183b9 None admin] PATH_INFO: `/v3/domains/a8b6e383e1c945608bfb70d4a9adb5b2` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:36:39.922678 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4067b5fb-692d-4368-bd00-497c212183b9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=a8b6e383e1c945608bfb70d4a9adb5b2)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:36:39.924100 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4067b5fb-692d-4368-bd00-497c212183b9 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:36:39.925672 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 602/1176] 10.222.0.36 () {56 vars in 1183 bytes} [Wed Jul 27 10:36:39 2022] GET /identity/v3/domains/a8b6e383e1c945608bfb70d4a9adb5b2 => generated 267 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:36:39.931829 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bae478-5bc5-44a2-8e0d-37e6b69db151 None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:36:39.983065 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:36:40.045488 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bae478-5bc5-44a2-8e0d-37e6b69db151 None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:36:40.082615 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c99da5a-2a3e-48fc-949e-891f82ae7c61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:36:40.083280 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c99da5a-2a3e-48fc-949e-891f82ae7c61 None admin] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:36:40.083567 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c99da5a-2a3e-48fc-949e-891f82ae7c61 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:36:40.083843 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c99da5a-2a3e-48fc-949e-891f82ae7c61 None admin] PATH_INFO: `/v3/users` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:36:40.084543 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c99da5a-2a3e-48fc-949e-891f82ae7c61 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:36:40.085978 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c99da5a-2a3e-48fc-949e-891f82ae7c61 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:36:40.106910 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 575/1177] 10.222.0.36 () {60 vars in 1132 bytes} [Wed Jul 27 10:36:39 2022] POST /identity/v3/users => generated 311 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 10:36:42.108685 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6491bd02-83e5-455c-83f5-de61f62c3386 None None] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:36:42.109224 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6491bd02-83e5-455c-83f5-de61f62c3386 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:36:42.109224 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6491bd02-83e5-455c-83f5-de61f62c3386 None None] PATH_INFO: `/` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:36:42.110611 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 603/1178] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 10:36:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 10:36:42.116356 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ef4c53-dc56-47f7-88d7-5cf61f39fb71 None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:36:42.116727 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ef4c53-dc56-47f7-88d7-5cf61f39fb71 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:36:42.116905 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ef4c53-dc56-47f7-88d7-5cf61f39fb71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:36:42.135518 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-94ef4c53-dc56-47f7-88d7-5cf61f39fb71 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:36:42.136697 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-94ef4c53-dc56-47f7-88d7-5cf61f39fb71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:36:42.200572 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 576/1179] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 10:36:42 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 10:36:42.223886 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6491bd02-83e5-455c-83f5-de61f62c3386 None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:36:42.323236 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6491bd02-83e5-455c-83f5-de61f62c3386 None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:36:42.350488 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:36:42.356737 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9f0b1f-dded-4a9c-a2c7-ebef53025a06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:36:42.357368 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9f0b1f-dded-4a9c-a2c7-ebef53025a06 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:36:42.357652 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9f0b1f-dded-4a9c-a2c7-ebef53025a06 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:36:42.357929 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9f0b1f-dded-4a9c-a2c7-ebef53025a06 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:36:42.368778 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9f0b1f-dded-4a9c-a2c7-ebef53025a06 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 10:36:42.373686 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application [None req-ad9f0b1f-dded-4a9c-a2c7-ebef53025a06 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 10:36:42.373686 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:36:42.373686 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:36:42.373686 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:36:42.373686 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:36:42.373686 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:36:42.373686 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:36:42.373686 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:36:42.373686 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:36:42.373686 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:36:42.373686 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:36:42.373686 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:36:42.373686 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 10:36:42.373686 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 10:36:42.373686 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 10:36:42.373686 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 10:36:42.374784 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 10:36:42.374784 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 10:36:42.374784 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 10:36:42.374784 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 10:36:42.374784 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:36:42.374784 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:36:42.374784 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:36:42.374784 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:36:42.374784 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:36:42.374784 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:36:42.374784 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:36:42.374784 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application with Lock( Jul 27 10:36:42.374784 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:36:42.374784 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:36:42.374784 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:36:42.374784 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:36:42.375974 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:36:42.375974 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:36:42.375974 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:36:42.375974 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:36:42.375974 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 10:36:42.375974 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 10:36:42.375974 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 10:36:42.375974 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 10:36:42.375974 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 10:36:42.375974 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 10:36:42.375974 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 10:36:42.375974 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Jul 27 10:36:42.375974 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 604/1180] 10.222.0.36 () {56 vars in 1096 bytes} [Wed Jul 27 10:36:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 10:36:42.380761 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ef4c53-dc56-47f7-88d7-5cf61f39fb71 None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:36:42.409606 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:36:42.477860 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ef4c53-dc56-47f7-88d7-5cf61f39fb71 None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:36:42.511178 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbb20d7-2784-4b16-8745-0b9801ec0517 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:36:42.511792 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbb20d7-2784-4b16-8745-0b9801ec0517 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:36:42.512076 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbb20d7-2784-4b16-8745-0b9801ec0517 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:36:42.512353 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbb20d7-2784-4b16-8745-0b9801ec0517 None admin] PATH_INFO: `/v3/roles` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:36:42.513066 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbb20d7-2784-4b16-8745-0b9801ec0517 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86828) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:36:42.513374 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbb20d7-2784-4b16-8745-0b9801ec0517 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:36:42.514782 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbb20d7-2784-4b16-8745-0b9801ec0517 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:36:42.522586 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 577/1181] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jul 27 10:36:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:36:42.528272 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9f0b1f-dded-4a9c-a2c7-ebef53025a06 None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:36:42.626644 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9f0b1f-dded-4a9c-a2c7-ebef53025a06 None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:36:42.659760 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccaf593-8b4e-4304-8ff1-a09abce0b91b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:36:42.660387 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccaf593-8b4e-4304-8ff1-a09abce0b91b None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:36:42.661229 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccaf593-8b4e-4304-8ff1-a09abce0b91b None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:36:42.661507 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccaf593-8b4e-4304-8ff1-a09abce0b91b None admin] PATH_INFO: `/v3/users/1ea9174a6a294884bd4cab7d03bed331` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:36:42.674377 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eccaf593-8b4e-4304-8ff1-a09abce0b91b None admin] RBAC: Authorizing `identity:get_user(user_id=1ea9174a6a294884bd4cab7d03bed331)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:36:42.675846 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eccaf593-8b4e-4304-8ff1-a09abce0b91b None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:36:42.677488 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 605/1182] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Jul 27 10:36:42 2022] GET /identity/v3/users/1ea9174a6a294884bd4cab7d03bed331 => generated 311 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:36:42.683147 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbb20d7-2784-4b16-8745-0b9801ec0517 None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:36:42.772207 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:36:42.781131 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbb20d7-2784-4b16-8745-0b9801ec0517 None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:36:42.813574 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc774441-715e-4e55-8766-fcbee1890b33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:36:42.814185 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc774441-715e-4e55-8766-fcbee1890b33 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:36:42.814469 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc774441-715e-4e55-8766-fcbee1890b33 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:36:42.814773 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc774441-715e-4e55-8766-fcbee1890b33 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:36:42.878607 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc774441-715e-4e55-8766-fcbee1890b33 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:36:42.880089 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc774441-715e-4e55-8766-fcbee1890b33 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:36:42.965922 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:36:42.975849 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 578/1183] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jul 27 10:36:42 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 27 10:36:42.981693 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccaf593-8b4e-4304-8ff1-a09abce0b91b None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:36:43.029442 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:36:43.079065 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccaf593-8b4e-4304-8ff1-a09abce0b91b None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:36:43.112795 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ac129d-6120-4992-a40b-d86ad31713b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:36:43.113421 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ac129d-6120-4992-a40b-d86ad31713b4 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:36:43.113730 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ac129d-6120-4992-a40b-d86ad31713b4 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:36:43.114007 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ac129d-6120-4992-a40b-d86ad31713b4 None admin] PATH_INFO: `/v3/projects/53f7d57159094963a1f33715d8126d00` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:36:43.115180 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ac129d-6120-4992-a40b-d86ad31713b4 None admin] RBAC: Authorizing `identity:get_project(project_id=53f7d57159094963a1f33715d8126d00)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:36:43.116601 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ac129d-6120-4992-a40b-d86ad31713b4 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:36:43.118241 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 606/1184] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Jul 27 10:36:42 2022] GET /identity/v3/projects/53f7d57159094963a1f33715d8126d00 => generated 361 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:36:43.124445 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc774441-715e-4e55-8766-fcbee1890b33 None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:36:43.221594 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc774441-715e-4e55-8766-fcbee1890b33 None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:36:43.253814 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8ee03e-aa7c-43e4-a6d4-d66bd5ade3b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:36:43.254962 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8ee03e-aa7c-43e4-a6d4-d66bd5ade3b6 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:36:43.255247 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8ee03e-aa7c-43e4-a6d4-d66bd5ade3b6 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:36:43.255521 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8ee03e-aa7c-43e4-a6d4-d66bd5ade3b6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:36:43.256309 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8ee03e-aa7c-43e4-a6d4-d66bd5ade3b6 None admin] RBAC: Adding query filter params (role.id=21156560eb6b436bb567a40b4976f3f4, scope.project.id=53f7d57159094963a1f33715d8126d00, user.id=1ea9174a6a294884bd4cab7d03bed331) {{(pid=86828) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:36:43.256620 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8ee03e-aa7c-43e4-a6d4-d66bd5ade3b6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:36:43.258286 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8ee03e-aa7c-43e4-a6d4-d66bd5ade3b6 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:36:43.263550 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 579/1185] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Jul 27 10:36:43 2022] GET /identity/v3/role_assignments?user.id=1ea9174a6a294884bd4cab7d03bed331&scope.project.id=53f7d57159094963a1f33715d8126d00&role.id=21156560eb6b436bb567a40b4976f3f4 => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 10:36:45.200595 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b1d452-a661-4c84-a0f7-cb5373ea1a52 None None] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:36:45.201136 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b1d452-a661-4c84-a0f7-cb5373ea1a52 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:36:45.201136 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b1d452-a661-4c84-a0f7-cb5373ea1a52 None None] PATH_INFO: `/` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:36:45.202417 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 607/1186] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 10:36:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 10:36:45.207645 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb2521f-5258-4e31-aa2b-6822794c0dd2 None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:36:45.208068 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb2521f-5258-4e31-aa2b-6822794c0dd2 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:36:45.208208 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb2521f-5258-4e31-aa2b-6822794c0dd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:36:45.225892 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-ffb2521f-5258-4e31-aa2b-6822794c0dd2 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:36:45.227098 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-ffb2521f-5258-4e31-aa2b-6822794c0dd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:36:45.276323 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:36:45.293455 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 580/1187] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 10:36:45 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 27 10:36:45.315900 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b1d452-a661-4c84-a0f7-cb5373ea1a52 None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:36:45.413018 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b1d452-a661-4c84-a0f7-cb5373ea1a52 None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:36:45.429058 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:36:45.449706 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2514b3-fbd2-4ee6-bf40-ae98943649a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:36:45.450339 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2514b3-fbd2-4ee6-bf40-ae98943649a2 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:36:45.450692 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2514b3-fbd2-4ee6-bf40-ae98943649a2 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:36:45.450920 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2514b3-fbd2-4ee6-bf40-ae98943649a2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:36:45.461564 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2514b3-fbd2-4ee6-bf40-ae98943649a2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 10:36:45.466544 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application [None req-9d2514b3-fbd2-4ee6-bf40-ae98943649a2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 10:36:45.466544 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:36:45.466544 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:36:45.466544 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:36:45.466544 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:36:45.466544 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:36:45.466544 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:36:45.466544 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:36:45.466544 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:36:45.466544 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:36:45.466544 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:36:45.466544 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:36:45.466544 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 10:36:45.466544 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 10:36:45.466544 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 10:36:45.466544 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 10:36:45.467730 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 10:36:45.467730 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 10:36:45.467730 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 10:36:45.467730 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 10:36:45.467730 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:36:45.467730 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:36:45.467730 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:36:45.467730 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:36:45.467730 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:36:45.467730 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:36:45.467730 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:36:45.467730 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application with Lock( Jul 27 10:36:45.467730 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:36:45.467730 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:36:45.467730 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:36:45.467730 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:36:45.468636 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:36:45.468636 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:36:45.468636 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:36:45.468636 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:36:45.468636 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 10:36:45.468636 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 10:36:45.468636 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 10:36:45.468636 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 10:36:45.468636 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 10:36:45.468636 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 10:36:45.468636 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 10:36:45.468636 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Jul 27 10:36:45.468636 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 608/1188] 10.222.0.36 () {56 vars in 1096 bytes} [Wed Jul 27 10:36:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 10:36:45.477240 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb2521f-5258-4e31-aa2b-6822794c0dd2 None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:36:45.574192 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb2521f-5258-4e31-aa2b-6822794c0dd2 None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:36:45.607033 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166547bd-cf3e-4784-ad08-b8100bd1a77e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:36:45.607645 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166547bd-cf3e-4784-ad08-b8100bd1a77e None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:36:45.607928 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166547bd-cf3e-4784-ad08-b8100bd1a77e None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:36:45.608204 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166547bd-cf3e-4784-ad08-b8100bd1a77e None admin] PATH_INFO: `/v3/roles` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:36:45.608912 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166547bd-cf3e-4784-ad08-b8100bd1a77e None admin] RBAC: Adding query filter params (name=admin) {{(pid=86828) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:36:45.609296 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166547bd-cf3e-4784-ad08-b8100bd1a77e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:36:45.610677 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166547bd-cf3e-4784-ad08-b8100bd1a77e None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:36:45.618422 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 581/1189] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jul 27 10:36:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:36:45.624085 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2514b3-fbd2-4ee6-bf40-ae98943649a2 None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:36:45.721694 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2514b3-fbd2-4ee6-bf40-ae98943649a2 None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:36:45.756751 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392074a0-c0df-466e-9cfa-2c1c10345c10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:36:45.757496 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392074a0-c0df-466e-9cfa-2c1c10345c10 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:36:45.757778 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392074a0-c0df-466e-9cfa-2c1c10345c10 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:36:45.758314 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392074a0-c0df-466e-9cfa-2c1c10345c10 None admin] PATH_INFO: `/v3/users/1ea9174a6a294884bd4cab7d03bed331` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:36:45.759505 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392074a0-c0df-466e-9cfa-2c1c10345c10 None admin] RBAC: Authorizing `identity:get_user(user_id=1ea9174a6a294884bd4cab7d03bed331)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:36:45.761047 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392074a0-c0df-466e-9cfa-2c1c10345c10 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:36:45.762806 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 609/1190] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Jul 27 10:36:45 2022] GET /identity/v3/users/1ea9174a6a294884bd4cab7d03bed331 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:36:45.770071 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166547bd-cf3e-4784-ad08-b8100bd1a77e None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:36:45.819220 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:36:45.869470 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166547bd-cf3e-4784-ad08-b8100bd1a77e None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:36:45.902432 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bd5079-8711-4e8e-87d6-a7e3e695a30f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:36:45.903100 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bd5079-8711-4e8e-87d6-a7e3e695a30f None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:36:45.903387 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bd5079-8711-4e8e-87d6-a7e3e695a30f None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:36:45.903665 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bd5079-8711-4e8e-87d6-a7e3e695a30f None admin] PATH_INFO: `/v3/projects/53f7d57159094963a1f33715d8126d00` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:36:45.904815 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bd5079-8711-4e8e-87d6-a7e3e695a30f None admin] RBAC: Authorizing `identity:get_project(project_id=53f7d57159094963a1f33715d8126d00)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:36:45.906347 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bd5079-8711-4e8e-87d6-a7e3e695a30f None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:36:45.908028 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 582/1191] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Jul 27 10:36:45 2022] GET /identity/v3/projects/53f7d57159094963a1f33715d8126d00 => generated 361 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:36:45.914029 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392074a0-c0df-466e-9cfa-2c1c10345c10 None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:36:45.959366 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:36:46.011833 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392074a0-c0df-466e-9cfa-2c1c10345c10 None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:36:46.044362 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be947784-52b3-4c50-bd88-352ef96aa26e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:36:46.045018 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be947784-52b3-4c50-bd88-352ef96aa26e None admin] REQUEST_METHOD: `PUT` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:36:46.045297 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be947784-52b3-4c50-bd88-352ef96aa26e None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:36:46.045567 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be947784-52b3-4c50-bd88-352ef96aa26e None admin] PATH_INFO: `/v3/projects/53f7d57159094963a1f33715d8126d00/users/1ea9174a6a294884bd4cab7d03bed331/roles/21156560eb6b436bb567a40b4976f3f4` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:36:46.047523 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be947784-52b3-4c50-bd88-352ef96aa26e None admin] RBAC: Authorizing `identity:create_grant(project_id=53f7d57159094963a1f33715d8126d00, user_id=1ea9174a6a294884bd4cab7d03bed331, role_id=21156560eb6b436bb567a40b4976f3f4)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:36:46.048947 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be947784-52b3-4c50-bd88-352ef96aa26e None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:36:46.061570 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 610/1192] 10.222.0.36 () {58 vars in 1439 bytes} [Wed Jul 27 10:36:45 2022] PUT /identity/v3/projects/53f7d57159094963a1f33715d8126d00/users/1ea9174a6a294884bd4cab7d03bed331/roles/21156560eb6b436bb567a40b4976f3f4 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:36:48.057363 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ec9e47-dcef-4e3e-a951-2b40fe25ee4e None None] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:36:48.057885 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ec9e47-dcef-4e3e-a951-2b40fe25ee4e None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:36:48.057885 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ec9e47-dcef-4e3e-a951-2b40fe25ee4e None None] PATH_INFO: `/` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:36:48.059268 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 583/1193] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 10:36:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 10:36:48.064410 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93325ab3-5df8-48ff-b39d-12b710308714 None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:36:48.064807 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93325ab3-5df8-48ff-b39d-12b710308714 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:36:48.064963 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93325ab3-5df8-48ff-b39d-12b710308714 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:36:48.083296 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-93325ab3-5df8-48ff-b39d-12b710308714 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:36:48.084919 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-93325ab3-5df8-48ff-b39d-12b710308714 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:36:48.166830 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 611/1194] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 10:36:48 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 10:36:48.190189 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ec9e47-dcef-4e3e-a951-2b40fe25ee4e None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:36:48.288521 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ec9e47-dcef-4e3e-a951-2b40fe25ee4e None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:36:48.304012 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:36:48.320789 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a45f31f-d7cf-40b5-a712-04bfd52da5c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:36:48.321413 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a45f31f-d7cf-40b5-a712-04bfd52da5c4 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:36:48.321692 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a45f31f-d7cf-40b5-a712-04bfd52da5c4 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:36:48.321963 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a45f31f-d7cf-40b5-a712-04bfd52da5c4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:36:48.332020 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a45f31f-d7cf-40b5-a712-04bfd52da5c4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 10:36:48.336829 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application [None req-7a45f31f-d7cf-40b5-a712-04bfd52da5c4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 10:36:48.336829 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:36:48.336829 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:36:48.336829 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:36:48.336829 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:36:48.336829 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:36:48.336829 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:36:48.336829 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:36:48.336829 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:36:48.336829 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:36:48.336829 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:36:48.336829 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:36:48.336829 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 10:36:48.336829 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 10:36:48.336829 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 10:36:48.336829 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 10:36:48.337912 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 10:36:48.337912 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 10:36:48.337912 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 10:36:48.337912 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 10:36:48.337912 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:36:48.337912 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:36:48.337912 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:36:48.337912 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:36:48.337912 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:36:48.337912 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:36:48.337912 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:36:48.337912 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application with Lock( Jul 27 10:36:48.337912 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:36:48.337912 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:36:48.337912 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:36:48.337912 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:36:48.338985 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:36:48.338985 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:36:48.338985 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:36:48.338985 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:36:48.338985 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 10:36:48.338985 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 10:36:48.338985 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 10:36:48.338985 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 10:36:48.338985 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 10:36:48.338985 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 10:36:48.338985 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 10:36:48.338985 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Jul 27 10:36:48.338985 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 584/1195] 10.222.0.36 () {56 vars in 1096 bytes} [Wed Jul 27 10:36:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 10:36:48.343922 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93325ab3-5df8-48ff-b39d-12b710308714 None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:36:48.376667 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:36:48.442327 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93325ab3-5df8-48ff-b39d-12b710308714 None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:36:48.475446 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bc954d-0366-4b1c-b844-d193875f4313 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:36:48.476056 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bc954d-0366-4b1c-b844-d193875f4313 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:36:48.476342 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bc954d-0366-4b1c-b844-d193875f4313 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:36:48.476617 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bc954d-0366-4b1c-b844-d193875f4313 None admin] PATH_INFO: `/v3/roles` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:36:48.477360 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bc954d-0366-4b1c-b844-d193875f4313 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86827) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:36:48.477666 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bc954d-0366-4b1c-b844-d193875f4313 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:36:48.479104 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bc954d-0366-4b1c-b844-d193875f4313 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:36:48.487093 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 612/1196] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jul 27 10:36:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:36:48.493427 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a45f31f-d7cf-40b5-a712-04bfd52da5c4 None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:36:48.590439 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a45f31f-d7cf-40b5-a712-04bfd52da5c4 None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:36:48.623663 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e19f57-8274-41ed-933a-ab4912040a78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:36:48.624296 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e19f57-8274-41ed-933a-ab4912040a78 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:36:48.624581 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e19f57-8274-41ed-933a-ab4912040a78 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:36:48.624875 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e19f57-8274-41ed-933a-ab4912040a78 None admin] PATH_INFO: `/v3/users/1ea9174a6a294884bd4cab7d03bed331` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:36:48.626042 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e19f57-8274-41ed-933a-ab4912040a78 None admin] RBAC: Authorizing `identity:get_user(user_id=1ea9174a6a294884bd4cab7d03bed331)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:36:48.627508 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e19f57-8274-41ed-933a-ab4912040a78 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:36:48.629616 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 585/1197] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Jul 27 10:36:48 2022] GET /identity/v3/users/1ea9174a6a294884bd4cab7d03bed331 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:36:48.635607 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bc954d-0366-4b1c-b844-d193875f4313 None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:36:48.728007 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:36:48.733951 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bc954d-0366-4b1c-b844-d193875f4313 None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:36:48.766945 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84578e5-df22-42f0-b769-c2c818343ad7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:36:48.767549 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84578e5-df22-42f0-b769-c2c818343ad7 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:36:48.767826 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84578e5-df22-42f0-b769-c2c818343ad7 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:36:48.768097 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84578e5-df22-42f0-b769-c2c818343ad7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:36:48.831911 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84578e5-df22-42f0-b769-c2c818343ad7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:36:48.833421 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84578e5-df22-42f0-b769-c2c818343ad7 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:36:48.920731 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:36:48.927621 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 613/1198] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jul 27 10:36:48 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 27 10:36:48.933535 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e19f57-8274-41ed-933a-ab4912040a78 None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:36:48.977869 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:36:49.031283 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e19f57-8274-41ed-933a-ab4912040a78 None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:36:49.064141 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5554ef84-d9a4-43a7-ba7d-094bcf9ed56d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:36:49.064775 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5554ef84-d9a4-43a7-ba7d-094bcf9ed56d None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:36:49.065057 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5554ef84-d9a4-43a7-ba7d-094bcf9ed56d None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:36:49.065333 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5554ef84-d9a4-43a7-ba7d-094bcf9ed56d None admin] PATH_INFO: `/v3/projects/53f7d57159094963a1f33715d8126d00` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:36:49.066480 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5554ef84-d9a4-43a7-ba7d-094bcf9ed56d None admin] RBAC: Authorizing `identity:get_project(project_id=53f7d57159094963a1f33715d8126d00)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:36:49.068121 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5554ef84-d9a4-43a7-ba7d-094bcf9ed56d None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:36:49.069788 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 586/1199] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Jul 27 10:36:48 2022] GET /identity/v3/projects/53f7d57159094963a1f33715d8126d00 => generated 361 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:36:49.076013 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84578e5-df22-42f0-b769-c2c818343ad7 None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:36:49.171980 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84578e5-df22-42f0-b769-c2c818343ad7 None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:36:49.204541 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6da03c7-6ff6-4282-99d9-ef34ad20e730 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:36:49.205141 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6da03c7-6ff6-4282-99d9-ef34ad20e730 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:36:49.205419 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6da03c7-6ff6-4282-99d9-ef34ad20e730 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:36:49.205687 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6da03c7-6ff6-4282-99d9-ef34ad20e730 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:36:49.206467 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6da03c7-6ff6-4282-99d9-ef34ad20e730 None admin] RBAC: Adding query filter params (role.id=21156560eb6b436bb567a40b4976f3f4, scope.project.id=53f7d57159094963a1f33715d8126d00, user.id=1ea9174a6a294884bd4cab7d03bed331) {{(pid=86827) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:36:49.206800 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6da03c7-6ff6-4282-99d9-ef34ad20e730 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:36:49.208238 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6da03c7-6ff6-4282-99d9-ef34ad20e730 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:36:49.213833 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 614/1200] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Jul 27 10:36:49 2022] GET /identity/v3/role_assignments?user.id=1ea9174a6a294884bd4cab7d03bed331&scope.project.id=53f7d57159094963a1f33715d8126d00&role.id=21156560eb6b436bb567a40b4976f3f4 => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:36:59.060072 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=44 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:36:59.211363 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=70 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:37:44.205715 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6761a202-6ea8-447c-89dc-c775e84cf62e None None] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:37:44.206848 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6761a202-6ea8-447c-89dc-c775e84cf62e None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:37:44.206848 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6761a202-6ea8-447c-89dc-c775e84cf62e None None] PATH_INFO: `/` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:37:44.208066 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 587/1201] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 10:37:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 10:37:44.217819 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85585277-ff8d-4508-99a7-f218c44d0959 None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:37:44.218321 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85585277-ff8d-4508-99a7-f218c44d0959 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:37:44.218579 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85585277-ff8d-4508-99a7-f218c44d0959 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:37:44.238493 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-85585277-ff8d-4508-99a7-f218c44d0959 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:37:44.239846 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-85585277-ff8d-4508-99a7-f218c44d0959 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:37:44.332538 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 615/1202] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 10:37:44 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 10:37:44.357152 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6761a202-6ea8-447c-89dc-c775e84cf62e None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:37:44.492154 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6761a202-6ea8-447c-89dc-c775e84cf62e None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:37:44.533712 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec99532-1f44-445b-9fab-c062adc6129f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:37:44.535092 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec99532-1f44-445b-9fab-c062adc6129f None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:37:44.535377 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec99532-1f44-445b-9fab-c062adc6129f None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:37:44.535649 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec99532-1f44-445b-9fab-c062adc6129f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:37:44.554307 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec99532-1f44-445b-9fab-c062adc6129f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:37:44.554307 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec99532-1f44-445b-9fab-c062adc6129f None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:37:44.571719 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application [None req-6ec99532-1f44-445b-9fab-c062adc6129f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 10:37:44.571719 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:37:44.571719 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:37:44.571719 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:37:44.571719 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:37:44.571719 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:37:44.571719 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:37:44.571719 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:37:44.571719 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:37:44.571719 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:37:44.571719 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:37:44.571719 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:37:44.571719 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 10:37:44.571719 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 10:37:44.571719 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 10:37:44.572835 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 10:37:44.572835 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:37:44.572835 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:37:44.572835 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:37:44.572835 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:37:44.572835 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:37:44.572835 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:37:44.572835 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:37:44.572835 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application with Lock( Jul 27 10:37:44.572835 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:37:44.572835 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:37:44.572835 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:37:44.572835 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:37:44.572835 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:37:44.572835 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:37:44.572835 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:37:44.572835 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:37:44.574100 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 10:37:44.574100 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 10:37:44.574100 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 10:37:44.574100 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Jul 27 10:37:44.574100 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 588/1203] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 10:37:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 10:37:44.579626 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85585277-ff8d-4508-99a7-f218c44d0959 None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:37:44.703096 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85585277-ff8d-4508-99a7-f218c44d0959 None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:37:44.748122 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:37:44.756065 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0bdcbf-024c-4320-bcc1-71836128a2b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:37:44.756750 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0bdcbf-024c-4320-bcc1-71836128a2b0 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:37:44.757092 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0bdcbf-024c-4320-bcc1-71836128a2b0 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:37:44.757377 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0bdcbf-024c-4320-bcc1-71836128a2b0 None admin] PATH_INFO: `/v3/domains` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:37:44.758464 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0bdcbf-024c-4320-bcc1-71836128a2b0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86827) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:37:44.758915 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0bdcbf-024c-4320-bcc1-71836128a2b0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:37:44.760451 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0bdcbf-024c-4320-bcc1-71836128a2b0 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:37:44.778417 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 616/1204] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 10:37:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 10:37:44.785906 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec99532-1f44-445b-9fab-c062adc6129f None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:37:44.863083 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:37:44.927546 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec99532-1f44-445b-9fab-c062adc6129f None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:37:44.975620 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0497ed74-106d-403b-ac9a-597d5d053b2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:37:44.976327 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0497ed74-106d-403b-ac9a-597d5d053b2d None admin] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:37:44.976659 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0497ed74-106d-403b-ac9a-597d5d053b2d None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:37:44.977029 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0497ed74-106d-403b-ac9a-597d5d053b2d None admin] PATH_INFO: `/v3/users` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:37:44.977784 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0497ed74-106d-403b-ac9a-597d5d053b2d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:37:44.979572 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0497ed74-106d-403b-ac9a-597d5d053b2d None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:37:45.010688 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 589/1205] 10.222.0.36 () {60 vars in 1132 bytes} [Wed Jul 27 10:37:44 2022] POST /identity/v3/users => generated 251 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 10:37:47.335451 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608fb84e-b354-4125-99bf-467525aa9796 None None] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:37:47.336014 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608fb84e-b354-4125-99bf-467525aa9796 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:37:47.336329 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608fb84e-b354-4125-99bf-467525aa9796 None None] PATH_INFO: `/` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:37:47.338472 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 617/1206] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 10:37:47 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 10:37:47.345069 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1550eb97-6de7-43f2-bbaf-e25c9ab48775 None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:37:47.345729 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1550eb97-6de7-43f2-bbaf-e25c9ab48775 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:37:47.345729 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1550eb97-6de7-43f2-bbaf-e25c9ab48775 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:37:47.367607 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-1550eb97-6de7-43f2-bbaf-e25c9ab48775 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:37:47.369817 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-1550eb97-6de7-43f2-bbaf-e25c9ab48775 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:37:47.451466 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 590/1207] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 10:37:47 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 10:37:47.477649 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608fb84e-b354-4125-99bf-467525aa9796 None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:37:47.612916 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608fb84e-b354-4125-99bf-467525aa9796 None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:37:47.652684 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d029c11-79e2-4349-92a3-6c89e7914c89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:37:47.653373 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d029c11-79e2-4349-92a3-6c89e7914c89 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:37:47.653662 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d029c11-79e2-4349-92a3-6c89e7914c89 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:37:47.653974 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d029c11-79e2-4349-92a3-6c89e7914c89 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:37:47.668336 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d029c11-79e2-4349-92a3-6c89e7914c89 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 10:37:47.673761 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application [None req-6d029c11-79e2-4349-92a3-6c89e7914c89 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 27 10:37:47.673761 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:37:47.673761 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:37:47.673761 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:37:47.673761 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:37:47.673761 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:37:47.673761 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:37:47.673761 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:37:47.673761 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:37:47.673761 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:37:47.673761 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:37:47.673761 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:37:47.673761 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 10:37:47.673761 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 10:37:47.673761 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 10:37:47.673761 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 10:37:47.676196 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 10:37:47.676196 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 10:37:47.676196 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 10:37:47.676196 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 10:37:47.676196 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:37:47.676196 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:37:47.676196 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:37:47.676196 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:37:47.676196 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:37:47.676196 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:37:47.676196 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:37:47.676196 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application with Lock( Jul 27 10:37:47.676196 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:37:47.676196 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:37:47.676196 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:37:47.676196 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:37:47.677443 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:37:47.677443 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:37:47.677443 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:37:47.677443 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:37:47.677443 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 10:37:47.677443 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 10:37:47.677443 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 10:37:47.677443 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 10:37:47.677443 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 10:37:47.677443 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 10:37:47.677443 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 27 10:37:47.677443 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Jul 27 10:37:47.677443 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 618/1208] 10.222.0.36 () {56 vars in 1102 bytes} [Wed Jul 27 10:37:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 10:37:47.682152 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1550eb97-6de7-43f2-bbaf-e25c9ab48775 None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:37:47.735384 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:37:47.821295 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1550eb97-6de7-43f2-bbaf-e25c9ab48775 None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:37:47.864099 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab4f884-a8d7-4505-8412-4f203a3080ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:37:47.864722 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab4f884-a8d7-4505-8412-4f203a3080ee None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:37:47.865137 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab4f884-a8d7-4505-8412-4f203a3080ee None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:37:47.865433 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab4f884-a8d7-4505-8412-4f203a3080ee None admin] PATH_INFO: `/v3/roles` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:37:47.866399 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab4f884-a8d7-4505-8412-4f203a3080ee None admin] RBAC: Adding query filter params (name=service) {{(pid=86828) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:37:47.866739 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab4f884-a8d7-4505-8412-4f203a3080ee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:37:47.868269 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab4f884-a8d7-4505-8412-4f203a3080ee None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:37:47.878734 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 591/1209] 10.222.0.36 () {56 vars in 1103 bytes} [Wed Jul 27 10:37:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 10:37:47.884556 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d029c11-79e2-4349-92a3-6c89e7914c89 None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:37:47.946060 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:37:48.010753 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d029c11-79e2-4349-92a3-6c89e7914c89 None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:37:48.051371 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb94ba2-c7a2-4cf9-9127-8eab63bf5c2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:37:48.055237 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb94ba2-c7a2-4cf9-9127-8eab63bf5c2d None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:37:48.055542 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb94ba2-c7a2-4cf9-9127-8eab63bf5c2d None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:37:48.055815 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb94ba2-c7a2-4cf9-9127-8eab63bf5c2d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:37:48.068723 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb94ba2-c7a2-4cf9-9127-8eab63bf5c2d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:37:48.070702 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb94ba2-c7a2-4cf9-9127-8eab63bf5c2d None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:37:48.091807 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application [None req-dcb94ba2-c7a2-4cf9-9127-8eab63bf5c2d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 10:37:48.091807 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:37:48.091807 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:37:48.091807 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:37:48.091807 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:37:48.091807 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:37:48.091807 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:37:48.091807 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:37:48.091807 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:37:48.091807 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:37:48.091807 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:37:48.091807 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:37:48.091807 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 10:37:48.091807 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 10:37:48.091807 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 10:37:48.092955 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 10:37:48.092955 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:37:48.092955 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:37:48.092955 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:37:48.092955 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:37:48.092955 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:37:48.092955 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:37:48.092955 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:37:48.092955 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application with Lock( Jul 27 10:37:48.092955 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:37:48.092955 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:37:48.092955 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:37:48.092955 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:37:48.092955 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:37:48.092955 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:37:48.092955 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:37:48.092955 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:37:48.094061 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 10:37:48.094061 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 10:37:48.094061 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 10:37:48.094061 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Jul 27 10:37:48.101062 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 619/1210] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 10:37:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 10:37:48.106935 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab4f884-a8d7-4505-8412-4f203a3080ee None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:37:48.234483 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:37:48.238390 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab4f884-a8d7-4505-8412-4f203a3080ee None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:37:48.273674 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f7b74f-4e98-449e-a57c-0fee02f3e007 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:37:48.274295 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f7b74f-4e98-449e-a57c-0fee02f3e007 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:37:48.274574 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f7b74f-4e98-449e-a57c-0fee02f3e007 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:37:48.274903 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f7b74f-4e98-449e-a57c-0fee02f3e007 None admin] PATH_INFO: `/v3/domains` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:37:48.275567 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f7b74f-4e98-449e-a57c-0fee02f3e007 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86828) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:37:48.275900 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f7b74f-4e98-449e-a57c-0fee02f3e007 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:37:48.277246 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f7b74f-4e98-449e-a57c-0fee02f3e007 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:37:48.290158 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 592/1211] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 10:37:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:37:48.295858 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb94ba2-c7a2-4cf9-9127-8eab63bf5c2d None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:37:48.407738 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb94ba2-c7a2-4cf9-9127-8eab63bf5c2d None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:37:48.409304 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:37:48.446446 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbbcf2e-7e74-4821-93b0-7483925f0760 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:37:48.447710 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbbcf2e-7e74-4821-93b0-7483925f0760 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:37:48.448167 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbbcf2e-7e74-4821-93b0-7483925f0760 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:37:48.448407 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbbcf2e-7e74-4821-93b0-7483925f0760 None admin] PATH_INFO: `/v3/users/placement` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:37:48.458265 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbbcf2e-7e74-4821-93b0-7483925f0760 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:37:48.459896 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbbcf2e-7e74-4821-93b0-7483925f0760 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:37:48.474425 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application [None req-bdbbcf2e-7e74-4821-93b0-7483925f0760 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 27 10:37:48.474425 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:37:48.474425 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:37:48.474425 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:37:48.474425 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:37:48.474425 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:37:48.474425 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:37:48.474425 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:37:48.474425 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:37:48.474425 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:37:48.474425 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:37:48.474425 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:37:48.474425 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 10:37:48.474425 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 10:37:48.474425 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 10:37:48.474425 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 10:37:48.475608 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:37:48.475608 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:37:48.475608 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 10:37:48.475608 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 10:37:48.475608 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 10:37:48.475608 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 10:37:48.475608 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:37:48.475608 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:37:48.475608 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:37:48.475608 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:37:48.475608 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:37:48.475608 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application with Lock( Jul 27 10:37:48.475608 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:37:48.475608 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:37:48.475608 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:37:48.475608 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:37:48.475608 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:37:48.475608 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:37:48.481513 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:37:48.481513 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:37:48.481513 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 10:37:48.481513 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 10:37:48.481513 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 10:37:48.481513 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 10:37:48.481513 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 10:37:48.481513 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 10:37:48.481513 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 27 10:37:48.481513 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Jul 27 10:37:48.481513 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 620/1212] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 10:37:48 2022] GET /identity/v3/users/placement => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 10:37:48.482235 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f7b74f-4e98-449e-a57c-0fee02f3e007 None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:37:48.605711 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f7b74f-4e98-449e-a57c-0fee02f3e007 None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:37:48.644425 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edeceab9-50e8-42b6-b80f-b4b301670e1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:37:48.645100 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edeceab9-50e8-42b6-b80f-b4b301670e1a None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:37:48.645431 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edeceab9-50e8-42b6-b80f-b4b301670e1a None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:37:48.645719 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edeceab9-50e8-42b6-b80f-b4b301670e1a None admin] PATH_INFO: `/v3/users` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:37:48.646612 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edeceab9-50e8-42b6-b80f-b4b301670e1a None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=86828) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:37:48.646972 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edeceab9-50e8-42b6-b80f-b4b301670e1a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:37:48.648584 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edeceab9-50e8-42b6-b80f-b4b301670e1a None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:37:48.670219 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 593/1213] 10.222.0.36 () {56 vars in 1143 bytes} [Wed Jul 27 10:37:48 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:37:48.676742 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbbcf2e-7e74-4821-93b0-7483925f0760 None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:37:48.803671 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbbcf2e-7e74-4821-93b0-7483925f0760 None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:37:48.843048 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ab4fae-ee7d-4e79-a4d6-b755ed86b315 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:37:48.843737 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ab4fae-ee7d-4e79-a4d6-b755ed86b315 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:37:48.844071 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ab4fae-ee7d-4e79-a4d6-b755ed86b315 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:37:48.844399 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ab4fae-ee7d-4e79-a4d6-b755ed86b315 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:37:48.880445 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:37:48.922811 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ab4fae-ee7d-4e79-a4d6-b755ed86b315 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:37:48.924319 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ab4fae-ee7d-4e79-a4d6-b755ed86b315 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:37:49.059590 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 621/1214] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jul 27 10:37:48 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 27 10:37:49.066691 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edeceab9-50e8-42b6-b80f-b4b301670e1a None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:37:49.091411 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:37:49.192547 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edeceab9-50e8-42b6-b80f-b4b301670e1a None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:37:49.234470 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c052d822-b216-40bd-93ec-ec1bcf03f63d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:37:49.235768 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c052d822-b216-40bd-93ec-ec1bcf03f63d None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:37:49.236118 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c052d822-b216-40bd-93ec-ec1bcf03f63d None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:37:49.236451 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c052d822-b216-40bd-93ec-ec1bcf03f63d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:37:49.250222 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c052d822-b216-40bd-93ec-ec1bcf03f63d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:37:49.252072 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c052d822-b216-40bd-93ec-ec1bcf03f63d None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:37:49.271368 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application [None req-c052d822-b216-40bd-93ec-ec1bcf03f63d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 10:37:49.271368 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:37:49.271368 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:37:49.271368 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:37:49.271368 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:37:49.271368 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:37:49.271368 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:37:49.271368 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:37:49.271368 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:37:49.271368 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:37:49.271368 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:37:49.271368 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:37:49.271368 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 10:37:49.271368 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 10:37:49.271368 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 10:37:49.272303 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 10:37:49.272303 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:37:49.272303 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:37:49.272303 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:37:49.272303 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:37:49.272303 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:37:49.272303 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:37:49.272303 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:37:49.272303 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application with Lock( Jul 27 10:37:49.272303 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:37:49.272303 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:37:49.272303 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:37:49.272303 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:37:49.272303 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:37:49.272303 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:37:49.272303 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:37:49.272303 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:37:49.273286 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 10:37:49.273286 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 10:37:49.273286 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 10:37:49.273286 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Jul 27 10:37:49.273286 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 594/1215] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 10:37:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 10:37:49.282816 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ab4fae-ee7d-4e79-a4d6-b755ed86b315 None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:37:49.353296 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:37:49.401491 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ab4fae-ee7d-4e79-a4d6-b755ed86b315 None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:37:49.439864 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdce55c2-7d0e-4b72-8f70-5194cda700eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:37:49.441486 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdce55c2-7d0e-4b72-8f70-5194cda700eb None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:37:49.441486 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdce55c2-7d0e-4b72-8f70-5194cda700eb None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:37:49.441823 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdce55c2-7d0e-4b72-8f70-5194cda700eb None admin] PATH_INFO: `/v3/domains` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:37:49.442273 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdce55c2-7d0e-4b72-8f70-5194cda700eb None admin] RBAC: Adding query filter params (name=Default) {{(pid=86827) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:37:49.443041 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdce55c2-7d0e-4b72-8f70-5194cda700eb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:37:49.444288 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdce55c2-7d0e-4b72-8f70-5194cda700eb None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:37:49.458152 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 622/1216] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 10:37:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:37:49.463961 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c052d822-b216-40bd-93ec-ec1bcf03f63d None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:37:49.556907 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:37:49.591825 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c052d822-b216-40bd-93ec-ec1bcf03f63d None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:37:49.637246 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2dfad69-5153-4ad0-8359-5da84c4e3089 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:37:49.638049 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dfad69-5153-4ad0-8359-5da84c4e3089 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:37:49.638333 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dfad69-5153-4ad0-8359-5da84c4e3089 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:37:49.638818 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dfad69-5153-4ad0-8359-5da84c4e3089 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:37:49.647210 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2dfad69-5153-4ad0-8359-5da84c4e3089 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:37:49.648857 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2dfad69-5153-4ad0-8359-5da84c4e3089 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:37:49.660665 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application [None req-b2dfad69-5153-4ad0-8359-5da84c4e3089 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 10:37:49.660665 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:37:49.660665 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:37:49.660665 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:37:49.660665 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:37:49.660665 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:37:49.660665 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:37:49.660665 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:37:49.660665 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:37:49.660665 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:37:49.660665 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:37:49.660665 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:37:49.660665 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 10:37:49.660665 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 10:37:49.660665 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 10:37:49.661649 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 10:37:49.661649 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:37:49.661649 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:37:49.661649 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:37:49.661649 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:37:49.661649 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:37:49.661649 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:37:49.661649 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:37:49.661649 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application with Lock( Jul 27 10:37:49.661649 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:37:49.661649 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:37:49.661649 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:37:49.661649 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:37:49.661649 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:37:49.661649 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:37:49.661649 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:37:49.661649 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:37:49.663176 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 10:37:49.663176 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 10:37:49.663176 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 10:37:49.663176 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 10:37:49.663176 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 10:37:49.663176 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 10:37:49.663176 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 10:37:49.663176 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Jul 27 10:37:49.663176 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 595/1217] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Jul 27 10:37:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 10:37:49.667836 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdce55c2-7d0e-4b72-8f70-5194cda700eb None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:37:49.784921 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdce55c2-7d0e-4b72-8f70-5194cda700eb None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:37:49.820445 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:37:49.834445 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef2972a-3b2e-4553-9f64-dd51532a7c21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:37:49.835130 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef2972a-3b2e-4553-9f64-dd51532a7c21 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:37:49.835415 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef2972a-3b2e-4553-9f64-dd51532a7c21 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:37:49.835692 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef2972a-3b2e-4553-9f64-dd51532a7c21 None admin] PATH_INFO: `/v3/projects` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:37:49.836450 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef2972a-3b2e-4553-9f64-dd51532a7c21 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86827) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:37:49.836763 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef2972a-3b2e-4553-9f64-dd51532a7c21 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:37:49.838200 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef2972a-3b2e-4553-9f64-dd51532a7c21 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:37:49.856944 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 623/1218] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Jul 27 10:37:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 10:37:49.865434 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2dfad69-5153-4ad0-8359-5da84c4e3089 None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:37:50.011073 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2dfad69-5153-4ad0-8359-5da84c4e3089 None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:37:50.046254 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:37:50.054225 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce80e72a-0389-4f3f-8fb9-b2dda06f45c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:37:50.054880 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce80e72a-0389-4f3f-8fb9-b2dda06f45c4 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:37:50.055194 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce80e72a-0389-4f3f-8fb9-b2dda06f45c4 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:37:50.055530 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce80e72a-0389-4f3f-8fb9-b2dda06f45c4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:37:50.056211 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce80e72a-0389-4f3f-8fb9-b2dda06f45c4 None admin] RBAC: Adding query filter params (role.id=9e0038c6bbd34556af3f84f9e2a7c2c5, scope.project.id=5f296d8e3b8f4001909addc49d8dac01, user.id=16cc6179e9a04b6398fbacb40ad1c555) {{(pid=86828) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:37:50.056514 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce80e72a-0389-4f3f-8fb9-b2dda06f45c4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:37:50.057961 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce80e72a-0389-4f3f-8fb9-b2dda06f45c4 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:37:50.064090 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 596/1219] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Jul 27 10:37:49 2022] GET /identity/v3/role_assignments?user.id=16cc6179e9a04b6398fbacb40ad1c555&scope.project.id=5f296d8e3b8f4001909addc49d8dac01&role.id=9e0038c6bbd34556af3f84f9e2a7c2c5 => generated 259 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:37:52.333190 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55020db1-6c7d-4bcf-8e9e-c8b576e36bce None None] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:37:52.333816 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55020db1-6c7d-4bcf-8e9e-c8b576e36bce None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:37:52.333963 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55020db1-6c7d-4bcf-8e9e-c8b576e36bce None None] PATH_INFO: `/` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:37:52.335870 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 624/1220] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 10:37:52 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 10:37:52.341667 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4317ddb4-bd1e-44b4-aec0-e2980d8c568c None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:37:52.342143 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4317ddb4-bd1e-44b4-aec0-e2980d8c568c None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:37:52.342238 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4317ddb4-bd1e-44b4-aec0-e2980d8c568c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:37:52.361716 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-4317ddb4-bd1e-44b4-aec0-e2980d8c568c None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:37:52.363025 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-4317ddb4-bd1e-44b4-aec0-e2980d8c568c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:37:52.434735 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 597/1221] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 10:37:52 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 27 10:37:52.457447 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55020db1-6c7d-4bcf-8e9e-c8b576e36bce None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:37:52.569789 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55020db1-6c7d-4bcf-8e9e-c8b576e36bce None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:37:52.608853 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60930c71-2b5d-4293-8dc0-af7e777a313c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:37:52.609612 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60930c71-2b5d-4293-8dc0-af7e777a313c None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:37:52.610024 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60930c71-2b5d-4293-8dc0-af7e777a313c None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:37:52.610429 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60930c71-2b5d-4293-8dc0-af7e777a313c None admin] PATH_INFO: `/v3/roles/service` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:37:52.622879 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60930c71-2b5d-4293-8dc0-af7e777a313c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 10:37:52.628813 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application [None req-60930c71-2b5d-4293-8dc0-af7e777a313c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 27 10:37:52.628813 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:37:52.628813 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:37:52.628813 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:37:52.628813 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:37:52.628813 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:37:52.628813 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:37:52.628813 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:37:52.628813 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:37:52.628813 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:37:52.628813 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:37:52.628813 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:37:52.628813 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 10:37:52.628813 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 10:37:52.628813 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 10:37:52.628813 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 10:37:52.630041 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 10:37:52.630041 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 10:37:52.630041 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 10:37:52.630041 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 10:37:52.630041 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:37:52.630041 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:37:52.630041 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:37:52.630041 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:37:52.630041 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:37:52.630041 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:37:52.630041 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:37:52.630041 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application with Lock( Jul 27 10:37:52.630041 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:37:52.630041 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:37:52.630041 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:37:52.630041 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:37:52.630900 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:37:52.630900 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:37:52.630900 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:37:52.630900 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:37:52.630900 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 10:37:52.630900 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 10:37:52.630900 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 10:37:52.630900 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 10:37:52.630900 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 10:37:52.630900 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 10:37:52.630900 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 27 10:37:52.630900 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Jul 27 10:37:52.630900 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 625/1222] 10.222.0.36 () {56 vars in 1102 bytes} [Wed Jul 27 10:37:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 10:37:52.636410 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4317ddb4-bd1e-44b4-aec0-e2980d8c568c None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:37:52.743407 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4317ddb4-bd1e-44b4-aec0-e2980d8c568c None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:37:52.771698 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:37:52.783138 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c267c78-aa02-4a9f-8ef0-819053e539c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:37:52.784300 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c267c78-aa02-4a9f-8ef0-819053e539c1 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:37:52.784586 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c267c78-aa02-4a9f-8ef0-819053e539c1 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:37:52.784911 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c267c78-aa02-4a9f-8ef0-819053e539c1 None admin] PATH_INFO: `/v3/roles` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:37:52.785672 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c267c78-aa02-4a9f-8ef0-819053e539c1 None admin] RBAC: Adding query filter params (name=service) {{(pid=86828) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:37:52.786042 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c267c78-aa02-4a9f-8ef0-819053e539c1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:37:52.787487 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c267c78-aa02-4a9f-8ef0-819053e539c1 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:37:52.796360 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 598/1223] 10.222.0.36 () {56 vars in 1103 bytes} [Wed Jul 27 10:37:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:37:52.802617 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60930c71-2b5d-4293-8dc0-af7e777a313c None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:37:52.851732 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:37:52.922864 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60930c71-2b5d-4293-8dc0-af7e777a313c None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:37:52.964496 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ab91bc-bb5a-4255-aa2b-a0563e5c6f8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:37:52.965543 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ab91bc-bb5a-4255-aa2b-a0563e5c6f8e None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:37:52.965870 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ab91bc-bb5a-4255-aa2b-a0563e5c6f8e None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:37:52.966203 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ab91bc-bb5a-4255-aa2b-a0563e5c6f8e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:37:52.979310 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ab91bc-bb5a-4255-aa2b-a0563e5c6f8e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:37:52.981887 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ab91bc-bb5a-4255-aa2b-a0563e5c6f8e None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:37:52.999702 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application [None req-20ab91bc-bb5a-4255-aa2b-a0563e5c6f8e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 10:37:52.999702 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:37:52.999702 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:37:52.999702 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:37:52.999702 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:37:52.999702 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:37:52.999702 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:37:52.999702 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:37:52.999702 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:37:52.999702 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:37:52.999702 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:37:52.999702 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:37:52.999702 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 10:37:52.999702 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 10:37:52.999702 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 10:37:53.000761 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 10:37:53.000761 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:37:53.000761 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:37:53.000761 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:37:53.000761 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:37:53.000761 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:37:53.000761 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:37:53.000761 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:37:53.000761 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application with Lock( Jul 27 10:37:53.000761 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:37:53.000761 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:37:53.000761 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:37:53.000761 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:37:53.000761 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:37:53.000761 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:37:53.000761 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:37:53.000761 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:37:53.007068 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 10:37:53.007068 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 10:37:53.007068 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 10:37:53.007068 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Jul 27 10:37:53.007068 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 626/1224] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 10:37:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 10:37:53.008273 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c267c78-aa02-4a9f-8ef0-819053e539c1 None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:37:53.120316 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c267c78-aa02-4a9f-8ef0-819053e539c1 None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:37:53.155813 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c49de7-1a15-40a1-b600-632643b54e5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:37:53.156431 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c49de7-1a15-40a1-b600-632643b54e5e None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:37:53.156715 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c49de7-1a15-40a1-b600-632643b54e5e None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:37:53.156991 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c49de7-1a15-40a1-b600-632643b54e5e None admin] PATH_INFO: `/v3/domains` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:37:53.157683 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c49de7-1a15-40a1-b600-632643b54e5e None admin] RBAC: Adding query filter params (name=Default) {{(pid=86828) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:37:53.157988 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c49de7-1a15-40a1-b600-632643b54e5e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:37:53.159513 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c49de7-1a15-40a1-b600-632643b54e5e None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:37:53.172500 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 599/1225] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 10:37:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 10:37:53.179043 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ab91bc-bb5a-4255-aa2b-a0563e5c6f8e None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:37:53.286115 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:37:53.295037 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ab91bc-bb5a-4255-aa2b-a0563e5c6f8e None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:37:53.334462 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5f13a4-6240-4f9c-ab20-980352a0b8e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:37:53.335167 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5f13a4-6240-4f9c-ab20-980352a0b8e2 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:37:53.335435 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5f13a4-6240-4f9c-ab20-980352a0b8e2 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:37:53.335825 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5f13a4-6240-4f9c-ab20-980352a0b8e2 None admin] PATH_INFO: `/v3/users/placement` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:37:53.345831 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5f13a4-6240-4f9c-ab20-980352a0b8e2 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:37:53.348170 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5f13a4-6240-4f9c-ab20-980352a0b8e2 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:37:53.360506 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application [None req-ba5f13a4-6240-4f9c-ab20-980352a0b8e2 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 27 10:37:53.360506 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:37:53.360506 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:37:53.360506 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:37:53.360506 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:37:53.360506 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:37:53.360506 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:37:53.360506 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:37:53.360506 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:37:53.360506 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:37:53.360506 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:37:53.360506 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:37:53.360506 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 10:37:53.360506 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 10:37:53.360506 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 10:37:53.360506 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 10:37:53.362000 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:37:53.362000 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:37:53.362000 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 10:37:53.362000 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 10:37:53.362000 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 10:37:53.362000 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 10:37:53.362000 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:37:53.362000 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:37:53.362000 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:37:53.362000 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:37:53.362000 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:37:53.362000 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application with Lock( Jul 27 10:37:53.362000 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:37:53.362000 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:37:53.362000 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:37:53.362000 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:37:53.362000 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:37:53.362000 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:37:53.367072 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:37:53.367072 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:37:53.367072 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 10:37:53.367072 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 10:37:53.367072 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 10:37:53.367072 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 10:37:53.367072 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 10:37:53.367072 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 10:37:53.367072 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 27 10:37:53.367072 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Jul 27 10:37:53.367072 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 627/1226] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 10:37:53 2022] GET /identity/v3/users/placement => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 10:37:53.367890 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c49de7-1a15-40a1-b600-632643b54e5e None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:37:53.418636 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:37:53.478718 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c49de7-1a15-40a1-b600-632643b54e5e None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:37:53.516424 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964acd41-e0a4-4032-9fce-a91e64ecaa8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:37:53.517139 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964acd41-e0a4-4032-9fce-a91e64ecaa8a None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:37:53.517493 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964acd41-e0a4-4032-9fce-a91e64ecaa8a None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:37:53.517841 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964acd41-e0a4-4032-9fce-a91e64ecaa8a None admin] PATH_INFO: `/v3/users` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:37:53.519211 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964acd41-e0a4-4032-9fce-a91e64ecaa8a None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=86828) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:37:53.519518 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964acd41-e0a4-4032-9fce-a91e64ecaa8a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:37:53.520943 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964acd41-e0a4-4032-9fce-a91e64ecaa8a None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:37:53.539394 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 600/1227] 10.222.0.36 () {56 vars in 1143 bytes} [Wed Jul 27 10:37:53 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 10:37:53.545601 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5f13a4-6240-4f9c-ab20-980352a0b8e2 None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:37:53.666611 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5f13a4-6240-4f9c-ab20-980352a0b8e2 None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:37:53.707420 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54088dd-c2e7-4769-a5c6-9890e14b2abe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:37:53.708171 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54088dd-c2e7-4769-a5c6-9890e14b2abe None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:37:53.708541 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54088dd-c2e7-4769-a5c6-9890e14b2abe None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:37:53.708954 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54088dd-c2e7-4769-a5c6-9890e14b2abe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:37:53.721858 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54088dd-c2e7-4769-a5c6-9890e14b2abe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:37:53.723633 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54088dd-c2e7-4769-a5c6-9890e14b2abe None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:37:53.741660 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application [None req-d54088dd-c2e7-4769-a5c6-9890e14b2abe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 10:37:53.741660 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:37:53.741660 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:37:53.741660 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:37:53.741660 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:37:53.741660 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:37:53.741660 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:37:53.741660 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:37:53.741660 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:37:53.741660 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:37:53.741660 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:37:53.741660 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:37:53.741660 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 10:37:53.741660 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 10:37:53.741660 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 10:37:53.742697 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 10:37:53.742697 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:37:53.742697 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:37:53.742697 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:37:53.742697 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:37:53.742697 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:37:53.742697 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:37:53.742697 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:37:53.742697 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application with Lock( Jul 27 10:37:53.742697 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:37:53.742697 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:37:53.742697 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:37:53.742697 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:37:53.742697 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:37:53.742697 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:37:53.742697 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:37:53.742697 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:37:53.743626 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 10:37:53.743626 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 10:37:53.743626 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 10:37:53.743626 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Jul 27 10:37:53.743626 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 628/1228] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 10:37:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 10:37:53.748828 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964acd41-e0a4-4032-9fce-a91e64ecaa8a None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:37:53.866059 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964acd41-e0a4-4032-9fce-a91e64ecaa8a None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:37:53.868931 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:37:53.913259 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4a9d61-a11e-4319-88d8-7024b26b2650 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:37:53.913902 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4a9d61-a11e-4319-88d8-7024b26b2650 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:37:53.914183 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4a9d61-a11e-4319-88d8-7024b26b2650 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:37:53.914460 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4a9d61-a11e-4319-88d8-7024b26b2650 None admin] PATH_INFO: `/v3/domains` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:37:53.915193 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4a9d61-a11e-4319-88d8-7024b26b2650 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86828) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:37:53.915530 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4a9d61-a11e-4319-88d8-7024b26b2650 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:37:53.917054 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4a9d61-a11e-4319-88d8-7024b26b2650 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:37:53.931149 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 601/1229] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 10:37:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:37:53.938120 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54088dd-c2e7-4769-a5c6-9890e14b2abe None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:37:53.953207 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:37:54.065768 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54088dd-c2e7-4769-a5c6-9890e14b2abe None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:37:54.101209 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5161ef-1c77-4dd6-bff0-2e1e848acc31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:37:54.101931 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5161ef-1c77-4dd6-bff0-2e1e848acc31 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:37:54.102276 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5161ef-1c77-4dd6-bff0-2e1e848acc31 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:37:54.102650 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5161ef-1c77-4dd6-bff0-2e1e848acc31 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:37:54.110512 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5161ef-1c77-4dd6-bff0-2e1e848acc31 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:37:54.112655 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5161ef-1c77-4dd6-bff0-2e1e848acc31 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:37:54.125191 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application [None req-ab5161ef-1c77-4dd6-bff0-2e1e848acc31 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 10:37:54.125191 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:37:54.125191 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:37:54.125191 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:37:54.125191 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:37:54.125191 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:37:54.125191 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:37:54.125191 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:37:54.125191 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:37:54.125191 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:37:54.125191 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:37:54.125191 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:37:54.125191 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 10:37:54.125191 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 10:37:54.125191 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 10:37:54.126318 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 10:37:54.126318 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:37:54.126318 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:37:54.126318 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:37:54.126318 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:37:54.126318 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:37:54.126318 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:37:54.126318 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:37:54.126318 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application with Lock( Jul 27 10:37:54.126318 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:37:54.126318 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:37:54.126318 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:37:54.126318 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:37:54.126318 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:37:54.126318 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:37:54.126318 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:37:54.126318 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:37:54.127294 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 10:37:54.127294 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 10:37:54.127294 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 10:37:54.127294 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 10:37:54.127294 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 10:37:54.127294 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 10:37:54.127294 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 10:37:54.127294 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Jul 27 10:37:54.127294 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 629/1230] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Jul 27 10:37:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 10:37:54.133444 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4a9d61-a11e-4319-88d8-7024b26b2650 None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:37:54.244185 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4a9d61-a11e-4319-88d8-7024b26b2650 None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:37:54.287005 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00c340d-df67-4cdc-b244-cd5550aea2ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:37:54.287636 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00c340d-df67-4cdc-b244-cd5550aea2ad None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:37:54.287920 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00c340d-df67-4cdc-b244-cd5550aea2ad None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:37:54.288198 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00c340d-df67-4cdc-b244-cd5550aea2ad None admin] PATH_INFO: `/v3/projects` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:37:54.288952 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00c340d-df67-4cdc-b244-cd5550aea2ad None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86828) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:37:54.289267 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00c340d-df67-4cdc-b244-cd5550aea2ad None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:37:54.291014 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00c340d-df67-4cdc-b244-cd5550aea2ad None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:37:54.307608 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 602/1231] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Jul 27 10:37:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:37:54.314931 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5161ef-1c77-4dd6-bff0-2e1e848acc31 None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:37:54.392411 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:37:54.434085 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5161ef-1c77-4dd6-bff0-2e1e848acc31 None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:37:54.473644 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07234f08-bff9-4a17-a33c-dd8d98ffc6e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:37:54.474395 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07234f08-bff9-4a17-a33c-dd8d98ffc6e7 None admin] REQUEST_METHOD: `PUT` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:37:54.474622 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07234f08-bff9-4a17-a33c-dd8d98ffc6e7 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:37:54.474912 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07234f08-bff9-4a17-a33c-dd8d98ffc6e7 None admin] PATH_INFO: `/v3/projects/5f296d8e3b8f4001909addc49d8dac01/users/16cc6179e9a04b6398fbacb40ad1c555/roles/9e0038c6bbd34556af3f84f9e2a7c2c5` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:37:54.490502 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07234f08-bff9-4a17-a33c-dd8d98ffc6e7 None admin] RBAC: Authorizing `identity:create_grant(project_id=5f296d8e3b8f4001909addc49d8dac01, user_id=16cc6179e9a04b6398fbacb40ad1c555, role_id=9e0038c6bbd34556af3f84f9e2a7c2c5)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:37:54.492070 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07234f08-bff9-4a17-a33c-dd8d98ffc6e7 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:37:54.505453 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 630/1232] 10.222.0.36 () {58 vars in 1439 bytes} [Wed Jul 27 10:37:54 2022] PUT /identity/v3/projects/5f296d8e3b8f4001909addc49d8dac01/users/16cc6179e9a04b6398fbacb40ad1c555/roles/9e0038c6bbd34556af3f84f9e2a7c2c5 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:37:56.613604 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7aae01-fbd4-4888-ac0d-6762607a916e None None] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:37:56.614216 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7aae01-fbd4-4888-ac0d-6762607a916e None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:37:56.614216 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7aae01-fbd4-4888-ac0d-6762607a916e None None] PATH_INFO: `/` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:37:56.615942 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 603/1233] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 10:37:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 10:37:56.621452 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89245666-30e0-4954-a18a-ac0cd172694b None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:37:56.621887 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89245666-30e0-4954-a18a-ac0cd172694b None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:37:56.621997 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89245666-30e0-4954-a18a-ac0cd172694b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:37:56.642702 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-89245666-30e0-4954-a18a-ac0cd172694b None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:37:56.644063 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-89245666-30e0-4954-a18a-ac0cd172694b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:37:56.714693 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 631/1234] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 10:37:56 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 10:37:56.738938 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7aae01-fbd4-4888-ac0d-6762607a916e None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:37:56.766119 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:37:56.852234 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7aae01-fbd4-4888-ac0d-6762607a916e None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:37:56.904483 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6210eb-3db2-487c-a079-d61bd67ebb42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:37:56.905476 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6210eb-3db2-487c-a079-d61bd67ebb42 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:37:56.905950 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6210eb-3db2-487c-a079-d61bd67ebb42 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:37:56.906419 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6210eb-3db2-487c-a079-d61bd67ebb42 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:37:56.919714 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6210eb-3db2-487c-a079-d61bd67ebb42 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 10:37:56.925489 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application [None req-2c6210eb-3db2-487c-a079-d61bd67ebb42 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 27 10:37:56.925489 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:37:56.925489 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:37:56.925489 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:37:56.925489 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:37:56.925489 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:37:56.925489 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:37:56.925489 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:37:56.925489 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:37:56.925489 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:37:56.925489 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:37:56.925489 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:37:56.925489 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 10:37:56.925489 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 10:37:56.925489 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 10:37:56.925489 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 10:37:56.926486 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 10:37:56.926486 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 10:37:56.926486 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 10:37:56.926486 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 10:37:56.926486 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:37:56.926486 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:37:56.926486 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:37:56.926486 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:37:56.926486 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:37:56.926486 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:37:56.926486 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:37:56.926486 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application with Lock( Jul 27 10:37:56.926486 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:37:56.926486 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:37:56.926486 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:37:56.926486 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:37:56.929089 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:37:56.929089 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:37:56.929089 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:37:56.929089 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:37:56.929089 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 10:37:56.929089 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 10:37:56.929089 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 10:37:56.929089 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 10:37:56.929089 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 10:37:56.929089 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 10:37:56.929089 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 27 10:37:56.929089 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Jul 27 10:37:56.929089 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 604/1235] 10.222.0.36 () {56 vars in 1102 bytes} [Wed Jul 27 10:37:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 10:37:56.933702 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89245666-30e0-4954-a18a-ac0cd172694b None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:37:56.988592 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:37:57.045009 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89245666-30e0-4954-a18a-ac0cd172694b None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:37:57.079331 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5d6b16-ca5d-47de-bb5f-ba06fa2f9627 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:37:57.079948 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5d6b16-ca5d-47de-bb5f-ba06fa2f9627 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:37:57.080230 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5d6b16-ca5d-47de-bb5f-ba06fa2f9627 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:37:57.081032 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5d6b16-ca5d-47de-bb5f-ba06fa2f9627 None admin] PATH_INFO: `/v3/roles` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:37:57.081744 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5d6b16-ca5d-47de-bb5f-ba06fa2f9627 None admin] RBAC: Adding query filter params (name=service) {{(pid=86827) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:37:57.082050 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5d6b16-ca5d-47de-bb5f-ba06fa2f9627 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:37:57.083511 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5d6b16-ca5d-47de-bb5f-ba06fa2f9627 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:37:57.091780 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 632/1236] 10.222.0.36 () {56 vars in 1103 bytes} [Wed Jul 27 10:37:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:37:57.098126 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6210eb-3db2-487c-a079-d61bd67ebb42 None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:37:57.190668 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:37:57.209380 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6210eb-3db2-487c-a079-d61bd67ebb42 None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:37:57.255156 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe3adea-53f3-4e75-ba19-93256873b1bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:37:57.255909 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe3adea-53f3-4e75-ba19-93256873b1bd None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:37:57.256209 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe3adea-53f3-4e75-ba19-93256873b1bd None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:37:57.256536 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe3adea-53f3-4e75-ba19-93256873b1bd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:37:57.270304 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe3adea-53f3-4e75-ba19-93256873b1bd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:37:57.271849 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe3adea-53f3-4e75-ba19-93256873b1bd None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:37:57.289953 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application [None req-dbe3adea-53f3-4e75-ba19-93256873b1bd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 10:37:57.289953 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:37:57.289953 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:37:57.289953 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:37:57.289953 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:37:57.289953 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:37:57.289953 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:37:57.289953 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:37:57.289953 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:37:57.289953 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:37:57.289953 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:37:57.289953 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:37:57.289953 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 10:37:57.289953 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 10:37:57.289953 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 10:37:57.291208 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 10:37:57.291208 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:37:57.291208 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:37:57.291208 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:37:57.291208 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:37:57.291208 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:37:57.291208 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:37:57.291208 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:37:57.291208 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application with Lock( Jul 27 10:37:57.291208 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:37:57.291208 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:37:57.291208 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:37:57.291208 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:37:57.291208 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:37:57.291208 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:37:57.291208 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:37:57.291208 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:37:57.293073 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 10:37:57.293073 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 10:37:57.293073 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 10:37:57.293073 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Jul 27 10:37:57.293073 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 605/1237] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 10:37:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 10:37:57.297523 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5d6b16-ca5d-47de-bb5f-ba06fa2f9627 None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:37:57.423149 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5d6b16-ca5d-47de-bb5f-ba06fa2f9627 None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:37:57.432408 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:37:57.460007 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48e01a4-0079-4f54-bfcb-4636735a8c31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:37:57.460551 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48e01a4-0079-4f54-bfcb-4636735a8c31 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:37:57.460879 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48e01a4-0079-4f54-bfcb-4636735a8c31 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:37:57.461160 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48e01a4-0079-4f54-bfcb-4636735a8c31 None admin] PATH_INFO: `/v3/domains` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:37:57.461857 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48e01a4-0079-4f54-bfcb-4636735a8c31 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86827) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:37:57.462168 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48e01a4-0079-4f54-bfcb-4636735a8c31 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:37:57.463570 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48e01a4-0079-4f54-bfcb-4636735a8c31 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:37:57.476247 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 633/1238] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 10:37:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:37:57.483344 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe3adea-53f3-4e75-ba19-93256873b1bd None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:37:57.602729 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe3adea-53f3-4e75-ba19-93256873b1bd None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:37:57.647792 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83985758-6b6d-4e00-9ffe-1eddab949103 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:37:57.648645 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83985758-6b6d-4e00-9ffe-1eddab949103 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:37:57.648936 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83985758-6b6d-4e00-9ffe-1eddab949103 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:37:57.649205 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83985758-6b6d-4e00-9ffe-1eddab949103 None admin] PATH_INFO: `/v3/users/placement` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:37:57.652272 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:37:57.658088 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83985758-6b6d-4e00-9ffe-1eddab949103 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:37:57.659837 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83985758-6b6d-4e00-9ffe-1eddab949103 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:37:57.672215 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application [None req-83985758-6b6d-4e00-9ffe-1eddab949103 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 27 10:37:57.672215 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:37:57.672215 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:37:57.672215 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:37:57.672215 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:37:57.672215 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:37:57.672215 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:37:57.672215 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:37:57.672215 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:37:57.672215 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:37:57.672215 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:37:57.672215 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:37:57.672215 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 10:37:57.672215 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 10:37:57.672215 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 10:37:57.672215 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 10:37:57.673514 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:37:57.673514 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:37:57.673514 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 10:37:57.673514 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 10:37:57.673514 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 10:37:57.673514 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 10:37:57.673514 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:37:57.673514 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:37:57.673514 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:37:57.673514 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:37:57.673514 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:37:57.673514 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application with Lock( Jul 27 10:37:57.673514 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:37:57.673514 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:37:57.673514 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:37:57.673514 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:37:57.673514 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:37:57.673514 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:37:57.675452 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:37:57.675452 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:37:57.675452 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 10:37:57.675452 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 10:37:57.675452 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 10:37:57.675452 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 10:37:57.675452 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 10:37:57.675452 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 10:37:57.675452 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 27 10:37:57.675452 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Jul 27 10:37:57.675452 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 606/1239] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 10:37:57 2022] GET /identity/v3/users/placement => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 10:37:57.680286 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48e01a4-0079-4f54-bfcb-4636735a8c31 None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:37:57.797038 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48e01a4-0079-4f54-bfcb-4636735a8c31 None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:37:57.839037 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b24c1f-e374-4a6b-b57e-91c557725d7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:37:57.839726 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b24c1f-e374-4a6b-b57e-91c557725d7e None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:37:57.840038 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b24c1f-e374-4a6b-b57e-91c557725d7e None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:37:57.840390 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b24c1f-e374-4a6b-b57e-91c557725d7e None admin] PATH_INFO: `/v3/users` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:37:57.841478 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b24c1f-e374-4a6b-b57e-91c557725d7e None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=86827) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:37:57.841796 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b24c1f-e374-4a6b-b57e-91c557725d7e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:37:57.843770 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b24c1f-e374-4a6b-b57e-91c557725d7e None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:37:57.864837 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 634/1240] 10.222.0.36 () {56 vars in 1143 bytes} [Wed Jul 27 10:37:57 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:37:57.871298 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83985758-6b6d-4e00-9ffe-1eddab949103 None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:37:57.992744 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83985758-6b6d-4e00-9ffe-1eddab949103 None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:37:58.032971 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc19596-921c-4abf-ba45-33a987a2134c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:37:58.034274 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc19596-921c-4abf-ba45-33a987a2134c None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:37:58.034686 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc19596-921c-4abf-ba45-33a987a2134c None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:37:58.035029 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc19596-921c-4abf-ba45-33a987a2134c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:37:58.106071 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:37:58.114716 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc19596-921c-4abf-ba45-33a987a2134c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:37:58.116627 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc19596-921c-4abf-ba45-33a987a2134c None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:37:58.231748 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 607/1241] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jul 27 10:37:57 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 27 10:37:58.238131 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b24c1f-e374-4a6b-b57e-91c557725d7e None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:37:58.267678 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:37:58.358564 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b24c1f-e374-4a6b-b57e-91c557725d7e None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:37:58.400066 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf4138d-31bf-48b8-81e1-9f7fe4332605 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:37:58.400750 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf4138d-31bf-48b8-81e1-9f7fe4332605 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:37:58.401031 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf4138d-31bf-48b8-81e1-9f7fe4332605 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:37:58.401307 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf4138d-31bf-48b8-81e1-9f7fe4332605 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:37:58.413210 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf4138d-31bf-48b8-81e1-9f7fe4332605 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:37:58.414862 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf4138d-31bf-48b8-81e1-9f7fe4332605 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:37:58.433896 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application [None req-aaf4138d-31bf-48b8-81e1-9f7fe4332605 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 10:37:58.433896 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:37:58.433896 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:37:58.433896 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:37:58.433896 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:37:58.433896 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:37:58.433896 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:37:58.433896 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:37:58.433896 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:37:58.433896 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:37:58.433896 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:37:58.433896 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:37:58.433896 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 10:37:58.433896 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 10:37:58.433896 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 10:37:58.434956 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 10:37:58.434956 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:37:58.434956 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:37:58.434956 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:37:58.434956 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:37:58.434956 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:37:58.434956 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:37:58.434956 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:37:58.434956 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application with Lock( Jul 27 10:37:58.434956 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:37:58.434956 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:37:58.434956 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:37:58.434956 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:37:58.434956 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:37:58.434956 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:37:58.434956 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:37:58.434956 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:37:58.435831 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 10:37:58.435831 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 10:37:58.435831 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 10:37:58.435831 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Jul 27 10:37:58.435831 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 635/1242] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 10:37:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 10:37:58.442442 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc19596-921c-4abf-ba45-33a987a2134c None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:37:58.550206 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:37:58.557044 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc19596-921c-4abf-ba45-33a987a2134c None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:37:58.595000 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806d455d-ba4f-4daf-b392-d0fc14459d4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:37:58.595627 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806d455d-ba4f-4daf-b392-d0fc14459d4b None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:37:58.595911 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806d455d-ba4f-4daf-b392-d0fc14459d4b None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:37:58.596251 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806d455d-ba4f-4daf-b392-d0fc14459d4b None admin] PATH_INFO: `/v3/domains` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:37:58.596946 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806d455d-ba4f-4daf-b392-d0fc14459d4b None admin] RBAC: Adding query filter params (name=Default) {{(pid=86828) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:37:58.597254 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806d455d-ba4f-4daf-b392-d0fc14459d4b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:37:58.598917 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806d455d-ba4f-4daf-b392-d0fc14459d4b None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:37:58.614470 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 608/1243] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 10:37:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:37:58.620972 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf4138d-31bf-48b8-81e1-9f7fe4332605 None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:37:58.723018 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:37:58.747856 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf4138d-31bf-48b8-81e1-9f7fe4332605 None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:37:58.788090 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33dc42e-5f8b-4896-b19b-af7f0cd6159e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:37:58.788977 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33dc42e-5f8b-4896-b19b-af7f0cd6159e None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:37:58.789266 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33dc42e-5f8b-4896-b19b-af7f0cd6159e None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:37:58.789545 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33dc42e-5f8b-4896-b19b-af7f0cd6159e None admin] PATH_INFO: `/v3/projects/service` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:37:58.798278 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33dc42e-5f8b-4896-b19b-af7f0cd6159e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:37:58.799834 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33dc42e-5f8b-4896-b19b-af7f0cd6159e None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:37:58.813904 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application [None req-b33dc42e-5f8b-4896-b19b-af7f0cd6159e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 10:37:58.813904 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:37:58.813904 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:37:58.813904 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:37:58.813904 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:37:58.813904 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:37:58.813904 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:37:58.813904 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:37:58.813904 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:37:58.813904 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:37:58.813904 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:37:58.813904 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:37:58.813904 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 10:37:58.813904 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 10:37:58.813904 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 10:37:58.815949 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 10:37:58.815949 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:37:58.815949 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:37:58.815949 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:37:58.815949 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:37:58.815949 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:37:58.815949 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:37:58.815949 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:37:58.815949 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application with Lock( Jul 27 10:37:58.815949 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:37:58.815949 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:37:58.815949 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:37:58.815949 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:37:58.815949 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:37:58.815949 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:37:58.815949 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:37:58.815949 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:37:58.817046 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 10:37:58.817046 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 10:37:58.817046 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 10:37:58.817046 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 10:37:58.817046 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 10:37:58.817046 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 10:37:58.817046 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 10:37:58.817046 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Jul 27 10:37:58.817046 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 636/1244] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Jul 27 10:37:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 10:37:58.821318 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806d455d-ba4f-4daf-b392-d0fc14459d4b None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:37:58.935062 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806d455d-ba4f-4daf-b392-d0fc14459d4b None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:37:58.976544 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cccc332-4a7e-4a54-9479-42b7df4086fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:37:58.977240 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cccc332-4a7e-4a54-9479-42b7df4086fb None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:37:58.977521 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cccc332-4a7e-4a54-9479-42b7df4086fb None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:37:58.977812 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cccc332-4a7e-4a54-9479-42b7df4086fb None admin] PATH_INFO: `/v3/projects` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:37:58.978574 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cccc332-4a7e-4a54-9479-42b7df4086fb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86828) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:37:58.978966 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cccc332-4a7e-4a54-9479-42b7df4086fb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:37:58.980504 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cccc332-4a7e-4a54-9479-42b7df4086fb None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:37:58.993762 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 609/1245] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Jul 27 10:37:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 10:37:58.999929 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33dc42e-5f8b-4896-b19b-af7f0cd6159e None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:37:59.120408 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33dc42e-5f8b-4896-b19b-af7f0cd6159e None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:37:59.157666 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed008ad5-489b-4a7e-8d7e-8ec64666b968 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:37:59.158301 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed008ad5-489b-4a7e-8d7e-8ec64666b968 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:37:59.159305 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed008ad5-489b-4a7e-8d7e-8ec64666b968 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:37:59.159588 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed008ad5-489b-4a7e-8d7e-8ec64666b968 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:37:59.160407 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed008ad5-489b-4a7e-8d7e-8ec64666b968 None admin] RBAC: Adding query filter params (role.id=9e0038c6bbd34556af3f84f9e2a7c2c5, scope.project.id=5f296d8e3b8f4001909addc49d8dac01, user.id=16cc6179e9a04b6398fbacb40ad1c555) {{(pid=86827) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:37:59.160721 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed008ad5-489b-4a7e-8d7e-8ec64666b968 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:37:59.162224 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed008ad5-489b-4a7e-8d7e-8ec64666b968 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:37:59.168652 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 637/1246] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Jul 27 10:37:58 2022] GET /identity/v3/role_assignments?user.id=16cc6179e9a04b6398fbacb40ad1c555&scope.project.id=5f296d8e3b8f4001909addc49d8dac01&role.id=9e0038c6bbd34556af3f84f9e2a7c2c5 => generated 608 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:38:01.488164 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58da4c2-435f-4c25-82c8-19b29e147491 None None] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:38:01.489265 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58da4c2-435f-4c25-82c8-19b29e147491 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:38:01.489265 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58da4c2-435f-4c25-82c8-19b29e147491 None None] PATH_INFO: `/` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:38:01.490970 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 610/1247] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 10:38:01 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 10:38:01.496276 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da35e2d3-6e2c-48f7-9082-4e02bb89fa59 None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:38:01.496719 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da35e2d3-6e2c-48f7-9082-4e02bb89fa59 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:38:01.496827 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da35e2d3-6e2c-48f7-9082-4e02bb89fa59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:38:01.504366 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:38:01.518114 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-da35e2d3-6e2c-48f7-9082-4e02bb89fa59 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:38:01.519460 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-da35e2d3-6e2c-48f7-9082-4e02bb89fa59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:38:01.603495 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 638/1248] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 10:38:01 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 27 10:38:01.621423 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58da4c2-435f-4c25-82c8-19b29e147491 None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:38:01.644113 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:38:01.753457 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58da4c2-435f-4c25-82c8-19b29e147491 None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:38:01.795161 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79471f4-98af-43f3-866e-fdbf09e0744d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:38:01.795805 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79471f4-98af-43f3-866e-fdbf09e0744d None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:38:01.796088 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79471f4-98af-43f3-866e-fdbf09e0744d None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:38:01.796382 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79471f4-98af-43f3-866e-fdbf09e0744d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:38:01.808327 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79471f4-98af-43f3-866e-fdbf09e0744d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 10:38:01.813604 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application [None req-e79471f4-98af-43f3-866e-fdbf09e0744d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 10:38:01.813604 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:38:01.813604 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:38:01.813604 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:38:01.813604 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:38:01.813604 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:38:01.813604 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:38:01.813604 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:38:01.813604 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:38:01.813604 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:38:01.813604 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:38:01.813604 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:38:01.813604 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 10:38:01.813604 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 10:38:01.813604 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 10:38:01.813604 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 10:38:01.814734 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 10:38:01.814734 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 10:38:01.814734 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 10:38:01.814734 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 10:38:01.814734 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:38:01.814734 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:38:01.814734 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:38:01.814734 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:38:01.814734 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:38:01.814734 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:38:01.814734 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:38:01.814734 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application with Lock( Jul 27 10:38:01.814734 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:38:01.814734 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:38:01.814734 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:38:01.814734 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:38:01.815767 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:38:01.815767 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:38:01.815767 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:38:01.815767 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:38:01.815767 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 10:38:01.815767 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 10:38:01.815767 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 10:38:01.815767 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 10:38:01.815767 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 10:38:01.815767 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 10:38:01.815767 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 10:38:01.815767 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Jul 27 10:38:01.815767 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 611/1249] 10.222.0.36 () {56 vars in 1096 bytes} [Wed Jul 27 10:38:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 10:38:01.823034 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da35e2d3-6e2c-48f7-9082-4e02bb89fa59 None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:38:01.937915 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da35e2d3-6e2c-48f7-9082-4e02bb89fa59 None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:38:01.970855 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:38:01.977123 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafe6447-9a2e-459c-9c1f-8564114a0de0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:38:01.977747 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafe6447-9a2e-459c-9c1f-8564114a0de0 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:38:01.978030 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafe6447-9a2e-459c-9c1f-8564114a0de0 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:38:01.978342 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafe6447-9a2e-459c-9c1f-8564114a0de0 None admin] PATH_INFO: `/v3/roles` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:38:01.979084 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafe6447-9a2e-459c-9c1f-8564114a0de0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86827) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:38:01.979385 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafe6447-9a2e-459c-9c1f-8564114a0de0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:38:01.980745 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafe6447-9a2e-459c-9c1f-8564114a0de0 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:38:01.991298 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 639/1250] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jul 27 10:38:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:38:01.997020 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79471f4-98af-43f3-866e-fdbf09e0744d None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:38:02.089778 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:38:02.119003 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79471f4-98af-43f3-866e-fdbf09e0744d None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:38:02.160731 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28f8488-4d74-48cb-85c5-69c5bd341536 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:38:02.161529 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28f8488-4d74-48cb-85c5-69c5bd341536 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:38:02.161984 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28f8488-4d74-48cb-85c5-69c5bd341536 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:38:02.162302 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28f8488-4d74-48cb-85c5-69c5bd341536 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:38:02.175274 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28f8488-4d74-48cb-85c5-69c5bd341536 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:38:02.176724 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28f8488-4d74-48cb-85c5-69c5bd341536 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:38:02.193836 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application [None req-c28f8488-4d74-48cb-85c5-69c5bd341536 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 10:38:02.193836 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:38:02.193836 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:38:02.193836 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:38:02.193836 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:38:02.193836 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:38:02.193836 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:38:02.193836 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:38:02.193836 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:38:02.193836 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:38:02.193836 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:38:02.193836 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:38:02.193836 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 10:38:02.193836 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 10:38:02.193836 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 10:38:02.195119 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 10:38:02.195119 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:38:02.195119 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:38:02.195119 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:38:02.195119 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:38:02.195119 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:38:02.195119 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:38:02.195119 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:38:02.195119 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application with Lock( Jul 27 10:38:02.195119 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:38:02.195119 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:38:02.195119 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:38:02.195119 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:38:02.195119 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:38:02.195119 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:38:02.195119 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:38:02.195119 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:38:02.196721 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 10:38:02.196721 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 10:38:02.196721 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 10:38:02.196721 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Jul 27 10:38:02.197072 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 612/1251] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 10:38:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 10:38:02.203103 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafe6447-9a2e-459c-9c1f-8564114a0de0 None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:38:02.310751 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafe6447-9a2e-459c-9c1f-8564114a0de0 None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:38:02.347142 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcb93c9-acda-4f07-a552-5369eff515f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:38:02.347782 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcb93c9-acda-4f07-a552-5369eff515f5 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:38:02.348105 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcb93c9-acda-4f07-a552-5369eff515f5 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:38:02.348416 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcb93c9-acda-4f07-a552-5369eff515f5 None admin] PATH_INFO: `/v3/domains` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:38:02.349304 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bcb93c9-acda-4f07-a552-5369eff515f5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86827) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:38:02.349834 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bcb93c9-acda-4f07-a552-5369eff515f5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:38:02.352230 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bcb93c9-acda-4f07-a552-5369eff515f5 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:38:02.366763 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 640/1252] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 10:38:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 10:38:02.373304 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28f8488-4d74-48cb-85c5-69c5bd341536 None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:38:02.493631 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28f8488-4d74-48cb-85c5-69c5bd341536 None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:38:02.521284 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:38:02.530699 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7980ac5-4e9f-4746-b024-c92ccfbe1c48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:38:02.531422 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7980ac5-4e9f-4746-b024-c92ccfbe1c48 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:38:02.531730 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7980ac5-4e9f-4746-b024-c92ccfbe1c48 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:38:02.532087 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7980ac5-4e9f-4746-b024-c92ccfbe1c48 None admin] PATH_INFO: `/v3/users/placement` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:38:02.541351 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7980ac5-4e9f-4746-b024-c92ccfbe1c48 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:38:02.542828 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7980ac5-4e9f-4746-b024-c92ccfbe1c48 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:38:02.554571 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application [None req-e7980ac5-4e9f-4746-b024-c92ccfbe1c48 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 27 10:38:02.554571 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:38:02.554571 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:38:02.554571 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:38:02.554571 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:38:02.554571 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:38:02.554571 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:38:02.554571 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:38:02.554571 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:38:02.554571 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:38:02.554571 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:38:02.554571 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:38:02.554571 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 10:38:02.554571 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 10:38:02.554571 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 10:38:02.554571 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 10:38:02.556321 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:38:02.556321 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:38:02.556321 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 10:38:02.556321 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 10:38:02.556321 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 10:38:02.556321 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 10:38:02.556321 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:38:02.556321 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:38:02.556321 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:38:02.556321 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:38:02.556321 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:38:02.556321 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application with Lock( Jul 27 10:38:02.556321 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:38:02.556321 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:38:02.556321 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:38:02.556321 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:38:02.556321 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:38:02.556321 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:38:02.557685 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:38:02.557685 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:38:02.557685 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 10:38:02.557685 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 10:38:02.557685 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 10:38:02.557685 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 10:38:02.557685 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 10:38:02.557685 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 10:38:02.557685 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 27 10:38:02.557685 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Jul 27 10:38:02.557685 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 613/1253] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 10:38:02 2022] GET /identity/v3/users/placement => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 10:38:02.561772 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcb93c9-acda-4f07-a552-5369eff515f5 None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:38:02.623795 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:38:02.677773 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcb93c9-acda-4f07-a552-5369eff515f5 None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:38:02.730621 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71ca597-0bad-42da-8d05-2fc9068a2cd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:38:02.731822 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71ca597-0bad-42da-8d05-2fc9068a2cd5 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:38:02.732267 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71ca597-0bad-42da-8d05-2fc9068a2cd5 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:38:02.732808 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71ca597-0bad-42da-8d05-2fc9068a2cd5 None admin] PATH_INFO: `/v3/users` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:38:02.734386 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71ca597-0bad-42da-8d05-2fc9068a2cd5 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=86827) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:38:02.734961 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71ca597-0bad-42da-8d05-2fc9068a2cd5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:38:02.737088 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71ca597-0bad-42da-8d05-2fc9068a2cd5 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:38:02.761896 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 641/1254] 10.222.0.36 () {56 vars in 1143 bytes} [Wed Jul 27 10:38:02 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:38:02.769996 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7980ac5-4e9f-4746-b024-c92ccfbe1c48 None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:38:02.898774 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7980ac5-4e9f-4746-b024-c92ccfbe1c48 None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:38:02.955275 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bcf550-f5eb-46cf-a14f-5a027eeee41f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:38:02.955996 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bcf550-f5eb-46cf-a14f-5a027eeee41f None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:38:02.956259 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bcf550-f5eb-46cf-a14f-5a027eeee41f None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:38:02.956671 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bcf550-f5eb-46cf-a14f-5a027eeee41f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:38:03.014967 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:38:03.030311 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bcf550-f5eb-46cf-a14f-5a027eeee41f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:38:03.031888 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bcf550-f5eb-46cf-a14f-5a027eeee41f None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:38:03.144235 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 614/1255] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jul 27 10:38:02 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 27 10:38:03.150211 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71ca597-0bad-42da-8d05-2fc9068a2cd5 None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:38:03.267265 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71ca597-0bad-42da-8d05-2fc9068a2cd5 None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:38:03.282370 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:38:03.307329 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9bab0f-b821-41ab-8bfc-a682a1b3447d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:38:03.308315 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9bab0f-b821-41ab-8bfc-a682a1b3447d None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:38:03.308845 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9bab0f-b821-41ab-8bfc-a682a1b3447d None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:38:03.309187 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9bab0f-b821-41ab-8bfc-a682a1b3447d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:38:03.320739 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9bab0f-b821-41ab-8bfc-a682a1b3447d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:38:03.322336 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9bab0f-b821-41ab-8bfc-a682a1b3447d None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:38:03.337624 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application [None req-9f9bab0f-b821-41ab-8bfc-a682a1b3447d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 10:38:03.337624 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:38:03.337624 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:38:03.337624 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:38:03.337624 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:38:03.337624 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:38:03.337624 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:38:03.337624 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:38:03.337624 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:38:03.337624 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:38:03.337624 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:38:03.337624 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:38:03.337624 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 10:38:03.337624 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 10:38:03.337624 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 10:38:03.338567 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 10:38:03.338567 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:38:03.338567 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:38:03.338567 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:38:03.338567 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:38:03.338567 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:38:03.338567 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:38:03.338567 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:38:03.338567 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application with Lock( Jul 27 10:38:03.338567 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:38:03.338567 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:38:03.338567 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:38:03.338567 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:38:03.338567 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:38:03.338567 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:38:03.338567 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:38:03.338567 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:38:03.339754 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 10:38:03.339754 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 10:38:03.339754 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 10:38:03.339754 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Jul 27 10:38:03.339754 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 642/1256] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 10:38:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 10:38:03.344549 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bcf550-f5eb-46cf-a14f-5a027eeee41f None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:38:03.441844 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:38:03.456453 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bcf550-f5eb-46cf-a14f-5a027eeee41f None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:38:03.494535 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c0dc20-cef4-4fe0-ab5a-45bf4f53ed46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:38:03.495286 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c0dc20-cef4-4fe0-ab5a-45bf4f53ed46 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:38:03.495574 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c0dc20-cef4-4fe0-ab5a-45bf4f53ed46 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:38:03.495851 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c0dc20-cef4-4fe0-ab5a-45bf4f53ed46 None admin] PATH_INFO: `/v3/domains` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:38:03.496702 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c0dc20-cef4-4fe0-ab5a-45bf4f53ed46 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86828) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:38:03.497074 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c0dc20-cef4-4fe0-ab5a-45bf4f53ed46 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:38:03.498689 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c0dc20-cef4-4fe0-ab5a-45bf4f53ed46 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:38:03.511787 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 615/1257] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 10:38:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:38:03.517484 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9bab0f-b821-41ab-8bfc-a682a1b3447d None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:38:03.633975 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9bab0f-b821-41ab-8bfc-a682a1b3447d None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:38:03.683782 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8433974-0550-432f-9627-088e2401f182 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:38:03.684430 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8433974-0550-432f-9627-088e2401f182 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:38:03.684718 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8433974-0550-432f-9627-088e2401f182 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:38:03.685082 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8433974-0550-432f-9627-088e2401f182 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:38:03.692856 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8433974-0550-432f-9627-088e2401f182 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:38:03.694260 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8433974-0550-432f-9627-088e2401f182 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:38:03.708794 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application [None req-f8433974-0550-432f-9627-088e2401f182 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 10:38:03.708794 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:38:03.708794 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:38:03.708794 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:38:03.708794 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:38:03.708794 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:38:03.708794 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:38:03.708794 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:38:03.708794 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:38:03.708794 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:38:03.708794 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:38:03.708794 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:38:03.708794 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 10:38:03.708794 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 10:38:03.708794 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 10:38:03.709890 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 10:38:03.709890 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:38:03.709890 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:38:03.709890 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:38:03.709890 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:38:03.709890 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:38:03.709890 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:38:03.709890 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:38:03.709890 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application with Lock( Jul 27 10:38:03.709890 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:38:03.709890 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:38:03.709890 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:38:03.709890 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:38:03.709890 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:38:03.709890 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:38:03.709890 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:38:03.709890 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:38:03.710976 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 10:38:03.710976 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 10:38:03.710976 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 10:38:03.710976 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 10:38:03.710976 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 10:38:03.710976 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 10:38:03.710976 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 10:38:03.710976 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Jul 27 10:38:03.710976 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 643/1258] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Jul 27 10:38:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 10:38:03.717257 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c0dc20-cef4-4fe0-ab5a-45bf4f53ed46 None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:38:03.842870 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c0dc20-cef4-4fe0-ab5a-45bf4f53ed46 None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:38:03.882901 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff81e0b-29ec-4aba-b1a3-96e29bf45131 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:38:03.883561 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff81e0b-29ec-4aba-b1a3-96e29bf45131 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:38:03.883867 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff81e0b-29ec-4aba-b1a3-96e29bf45131 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:38:03.884166 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff81e0b-29ec-4aba-b1a3-96e29bf45131 None admin] PATH_INFO: `/v3/projects` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:38:03.885106 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff81e0b-29ec-4aba-b1a3-96e29bf45131 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86828) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:38:03.885453 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff81e0b-29ec-4aba-b1a3-96e29bf45131 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:38:03.887081 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff81e0b-29ec-4aba-b1a3-96e29bf45131 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:38:03.901727 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 616/1259] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Jul 27 10:38:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:38:03.909101 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8433974-0550-432f-9627-088e2401f182 None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:38:03.940087 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:38:04.023167 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8433974-0550-432f-9627-088e2401f182 None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:38:04.058809 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80c56cd-0b0f-4500-8632-ca1c8971cf40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:38:04.059449 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80c56cd-0b0f-4500-8632-ca1c8971cf40 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:38:04.059742 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80c56cd-0b0f-4500-8632-ca1c8971cf40 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:38:04.060024 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80c56cd-0b0f-4500-8632-ca1c8971cf40 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:38:04.060833 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80c56cd-0b0f-4500-8632-ca1c8971cf40 None admin] RBAC: Adding query filter params (role.id=21156560eb6b436bb567a40b4976f3f4, scope.project.id=5f296d8e3b8f4001909addc49d8dac01, user.id=16cc6179e9a04b6398fbacb40ad1c555) {{(pid=86827) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:38:04.061150 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80c56cd-0b0f-4500-8632-ca1c8971cf40 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:38:04.062646 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80c56cd-0b0f-4500-8632-ca1c8971cf40 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:38:04.069600 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 644/1260] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Jul 27 10:38:03 2022] GET /identity/v3/role_assignments?user.id=16cc6179e9a04b6398fbacb40ad1c555&scope.project.id=5f296d8e3b8f4001909addc49d8dac01&role.id=21156560eb6b436bb567a40b4976f3f4 => generated 259 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:38:06.071538 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c117cd01-d59d-4676-a011-6bac7fab80b4 None None] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:38:06.072072 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c117cd01-d59d-4676-a011-6bac7fab80b4 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:38:06.072072 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c117cd01-d59d-4676-a011-6bac7fab80b4 None None] PATH_INFO: `/` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:38:06.073435 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 617/1261] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 10:38:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 10:38:06.078706 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29c923e-9f2c-4d36-b445-d3df226d522c None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:38:06.079090 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29c923e-9f2c-4d36-b445-d3df226d522c None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:38:06.079259 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29c923e-9f2c-4d36-b445-d3df226d522c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:38:06.098633 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-e29c923e-9f2c-4d36-b445-d3df226d522c None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:38:06.099814 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-e29c923e-9f2c-4d36-b445-d3df226d522c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:38:06.164020 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 645/1262] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 10:38:06 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 27 10:38:06.185811 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c117cd01-d59d-4676-a011-6bac7fab80b4 None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:38:06.196598 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:38:06.285735 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c117cd01-d59d-4676-a011-6bac7fab80b4 None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:38:06.319633 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f25505-20fa-4031-8dda-0c611f1f81a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:38:06.320269 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f25505-20fa-4031-8dda-0c611f1f81a8 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:38:06.320551 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f25505-20fa-4031-8dda-0c611f1f81a8 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:38:06.320825 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f25505-20fa-4031-8dda-0c611f1f81a8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:38:06.331231 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f25505-20fa-4031-8dda-0c611f1f81a8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 10:38:06.336126 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application [None req-35f25505-20fa-4031-8dda-0c611f1f81a8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 10:38:06.336126 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:38:06.336126 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:38:06.336126 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:38:06.336126 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:38:06.336126 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:38:06.336126 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:38:06.336126 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:38:06.336126 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:38:06.336126 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:38:06.336126 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:38:06.336126 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:38:06.336126 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 10:38:06.336126 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 10:38:06.336126 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 10:38:06.336126 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 10:38:06.337201 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 10:38:06.337201 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 10:38:06.337201 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 10:38:06.337201 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 10:38:06.337201 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:38:06.337201 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:38:06.337201 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:38:06.337201 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:38:06.337201 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:38:06.337201 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:38:06.337201 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:38:06.337201 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application with Lock( Jul 27 10:38:06.337201 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:38:06.337201 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:38:06.337201 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:38:06.337201 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:38:06.338397 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:38:06.338397 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:38:06.338397 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:38:06.338397 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:38:06.338397 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 10:38:06.338397 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 10:38:06.338397 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 10:38:06.338397 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 10:38:06.338397 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 10:38:06.338397 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 10:38:06.338397 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 10:38:06.338397 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Jul 27 10:38:06.338397 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 618/1263] 10.222.0.36 () {56 vars in 1096 bytes} [Wed Jul 27 10:38:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 10:38:06.343282 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29c923e-9f2c-4d36-b445-d3df226d522c None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:38:06.360829 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:38:06.440639 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29c923e-9f2c-4d36-b445-d3df226d522c None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:38:06.473767 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f58b08-51ac-4be3-8148-4a0b5584e8cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:38:06.474326 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f58b08-51ac-4be3-8148-4a0b5584e8cc None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:38:06.474620 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f58b08-51ac-4be3-8148-4a0b5584e8cc None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:38:06.474925 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f58b08-51ac-4be3-8148-4a0b5584e8cc None admin] PATH_INFO: `/v3/roles` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:38:06.475639 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f58b08-51ac-4be3-8148-4a0b5584e8cc None admin] RBAC: Adding query filter params (name=admin) {{(pid=86827) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:38:06.475948 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f58b08-51ac-4be3-8148-4a0b5584e8cc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:38:06.477312 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f58b08-51ac-4be3-8148-4a0b5584e8cc None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:38:06.485135 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 646/1264] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jul 27 10:38:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:38:06.490755 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f25505-20fa-4031-8dda-0c611f1f81a8 None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:38:06.555559 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:38:06.588970 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f25505-20fa-4031-8dda-0c611f1f81a8 None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:38:06.622747 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4453e1b5-4d94-4c5c-82b2-8b71f0c701e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:38:06.623423 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4453e1b5-4d94-4c5c-82b2-8b71f0c701e4 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:38:06.623743 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4453e1b5-4d94-4c5c-82b2-8b71f0c701e4 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:38:06.624058 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4453e1b5-4d94-4c5c-82b2-8b71f0c701e4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:38:06.636635 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4453e1b5-4d94-4c5c-82b2-8b71f0c701e4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:38:06.638086 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4453e1b5-4d94-4c5c-82b2-8b71f0c701e4 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:38:06.654548 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application [None req-4453e1b5-4d94-4c5c-82b2-8b71f0c701e4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 10:38:06.654548 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:38:06.654548 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:38:06.654548 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:38:06.654548 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:38:06.654548 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:38:06.654548 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:38:06.654548 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:38:06.654548 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:38:06.654548 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:38:06.654548 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:38:06.654548 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:38:06.654548 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 10:38:06.654548 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 10:38:06.654548 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 10:38:06.655638 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 10:38:06.655638 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:38:06.655638 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:38:06.655638 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:38:06.655638 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:38:06.655638 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:38:06.655638 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:38:06.655638 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:38:06.655638 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application with Lock( Jul 27 10:38:06.655638 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:38:06.655638 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:38:06.655638 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:38:06.655638 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:38:06.655638 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:38:06.655638 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:38:06.655638 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:38:06.655638 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:38:06.656684 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 10:38:06.656684 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 10:38:06.656684 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 10:38:06.656684 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Jul 27 10:38:06.656684 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 619/1265] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 10:38:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 10:38:06.662307 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f58b08-51ac-4be3-8148-4a0b5584e8cc None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:38:06.748817 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:38:06.768803 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f58b08-51ac-4be3-8148-4a0b5584e8cc None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:38:06.801828 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7a2611-d9c3-4d02-b2b5-ce0edb8f44cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:38:06.802441 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7a2611-d9c3-4d02-b2b5-ce0edb8f44cb None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:38:06.802762 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7a2611-d9c3-4d02-b2b5-ce0edb8f44cb None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:38:06.803041 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7a2611-d9c3-4d02-b2b5-ce0edb8f44cb None admin] PATH_INFO: `/v3/domains` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:38:06.803730 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7a2611-d9c3-4d02-b2b5-ce0edb8f44cb None admin] RBAC: Adding query filter params (name=Default) {{(pid=86827) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:38:06.804037 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7a2611-d9c3-4d02-b2b5-ce0edb8f44cb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:38:06.805429 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7a2611-d9c3-4d02-b2b5-ce0edb8f44cb None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:38:06.817335 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 647/1266] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 10:38:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:38:06.823244 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4453e1b5-4d94-4c5c-82b2-8b71f0c701e4 None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:38:06.919070 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4453e1b5-4d94-4c5c-82b2-8b71f0c701e4 None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:38:06.938329 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:38:06.953547 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2afc17e-6c49-4611-824b-5c07ed85403f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:38:06.954190 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2afc17e-6c49-4611-824b-5c07ed85403f None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:38:06.954474 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2afc17e-6c49-4611-824b-5c07ed85403f None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:38:06.954794 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2afc17e-6c49-4611-824b-5c07ed85403f None admin] PATH_INFO: `/v3/users/placement` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:38:06.962248 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2afc17e-6c49-4611-824b-5c07ed85403f None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:38:06.963719 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2afc17e-6c49-4611-824b-5c07ed85403f None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:38:06.974733 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application [None req-b2afc17e-6c49-4611-824b-5c07ed85403f None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 27 10:38:06.974733 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:38:06.974733 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:38:06.974733 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:38:06.974733 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:38:06.974733 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:38:06.974733 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:38:06.974733 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:38:06.974733 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:38:06.974733 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:38:06.974733 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:38:06.974733 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:38:06.974733 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 10:38:06.974733 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 10:38:06.974733 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 10:38:06.974733 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 10:38:06.975651 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:38:06.975651 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:38:06.975651 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 10:38:06.975651 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 10:38:06.975651 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 10:38:06.975651 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 10:38:06.975651 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:38:06.975651 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:38:06.975651 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:38:06.975651 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:38:06.975651 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:38:06.975651 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application with Lock( Jul 27 10:38:06.975651 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:38:06.975651 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:38:06.975651 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:38:06.975651 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:38:06.975651 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:38:06.975651 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:38:06.976646 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:38:06.976646 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:38:06.976646 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 10:38:06.976646 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 10:38:06.976646 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 10:38:06.976646 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 10:38:06.976646 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 10:38:06.976646 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 10:38:06.976646 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 27 10:38:06.976646 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Jul 27 10:38:06.976646 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 620/1267] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 10:38:06 2022] GET /identity/v3/users/placement => generated 87 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 10:38:06.981619 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7a2611-d9c3-4d02-b2b5-ce0edb8f44cb None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:38:07.078700 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7a2611-d9c3-4d02-b2b5-ce0edb8f44cb None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:38:07.111243 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8d3b39-f9e5-46ea-a69a-c19e5d0725b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:38:07.112448 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8d3b39-f9e5-46ea-a69a-c19e5d0725b3 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:38:07.112728 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8d3b39-f9e5-46ea-a69a-c19e5d0725b3 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:38:07.112999 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8d3b39-f9e5-46ea-a69a-c19e5d0725b3 None admin] PATH_INFO: `/v3/users` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:38:07.113779 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8d3b39-f9e5-46ea-a69a-c19e5d0725b3 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=86827) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:38:07.114084 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8d3b39-f9e5-46ea-a69a-c19e5d0725b3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:38:07.115520 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8d3b39-f9e5-46ea-a69a-c19e5d0725b3 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:38:07.123885 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:38:07.132849 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 648/1268] 10.222.0.36 () {56 vars in 1143 bytes} [Wed Jul 27 10:38:06 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:38:07.138339 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2afc17e-6c49-4611-824b-5c07ed85403f None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:38:07.235021 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2afc17e-6c49-4611-824b-5c07ed85403f None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:38:07.267631 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06bd0da-48dd-4be1-bbdb-5314a3174589 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:38:07.268286 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06bd0da-48dd-4be1-bbdb-5314a3174589 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:38:07.268571 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06bd0da-48dd-4be1-bbdb-5314a3174589 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:38:07.268848 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06bd0da-48dd-4be1-bbdb-5314a3174589 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:38:07.279094 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06bd0da-48dd-4be1-bbdb-5314a3174589 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:38:07.280536 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06bd0da-48dd-4be1-bbdb-5314a3174589 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:38:07.294957 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application [None req-d06bd0da-48dd-4be1-bbdb-5314a3174589 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 10:38:07.294957 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:38:07.294957 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:38:07.294957 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:38:07.294957 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:38:07.294957 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:38:07.294957 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:38:07.294957 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:38:07.294957 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:38:07.294957 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:38:07.294957 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:38:07.294957 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:38:07.294957 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 10:38:07.294957 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 10:38:07.294957 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 10:38:07.296017 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 10:38:07.296017 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:38:07.296017 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:38:07.296017 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:38:07.296017 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:38:07.296017 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:38:07.296017 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:38:07.296017 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:38:07.296017 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application with Lock( Jul 27 10:38:07.296017 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:38:07.296017 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:38:07.296017 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:38:07.296017 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:38:07.296017 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:38:07.296017 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:38:07.296017 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:38:07.296017 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:38:07.296974 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 10:38:07.296974 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 10:38:07.296974 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 10:38:07.296974 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Jul 27 10:38:07.296974 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 621/1269] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 10:38:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 10:38:07.302204 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8d3b39-f9e5-46ea-a69a-c19e5d0725b3 None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:38:07.399274 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8d3b39-f9e5-46ea-a69a-c19e5d0725b3 None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:38:07.432331 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c017a6-9434-4353-8b2b-344114abc6f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:38:07.432944 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c017a6-9434-4353-8b2b-344114abc6f1 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:38:07.433229 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c017a6-9434-4353-8b2b-344114abc6f1 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:38:07.433505 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c017a6-9434-4353-8b2b-344114abc6f1 None admin] PATH_INFO: `/v3/domains` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:38:07.434194 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c017a6-9434-4353-8b2b-344114abc6f1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86827) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:38:07.434499 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c017a6-9434-4353-8b2b-344114abc6f1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:38:07.435894 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c017a6-9434-4353-8b2b-344114abc6f1 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:38:07.446829 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 649/1270] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 10:38:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:38:07.452422 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06bd0da-48dd-4be1-bbdb-5314a3174589 None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:38:07.484064 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:38:07.551495 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06bd0da-48dd-4be1-bbdb-5314a3174589 None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:38:07.583688 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3be56de-1343-4113-ada6-d2622b51114e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:38:07.584318 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3be56de-1343-4113-ada6-d2622b51114e None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:38:07.584603 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3be56de-1343-4113-ada6-d2622b51114e None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:38:07.584887 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3be56de-1343-4113-ada6-d2622b51114e None admin] PATH_INFO: `/v3/projects/service` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:38:07.591353 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3be56de-1343-4113-ada6-d2622b51114e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:38:07.592823 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3be56de-1343-4113-ada6-d2622b51114e None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:38:07.606151 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application [None req-e3be56de-1343-4113-ada6-d2622b51114e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 10:38:07.606151 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:38:07.606151 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:38:07.606151 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:38:07.606151 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:38:07.606151 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:38:07.606151 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:38:07.606151 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:38:07.606151 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:38:07.606151 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:38:07.606151 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:38:07.606151 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:38:07.606151 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 10:38:07.606151 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 10:38:07.606151 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 10:38:07.607729 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 10:38:07.607729 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:38:07.607729 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:38:07.607729 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:38:07.607729 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:38:07.607729 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:38:07.607729 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:38:07.607729 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:38:07.607729 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application with Lock( Jul 27 10:38:07.607729 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:38:07.607729 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:38:07.607729 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:38:07.607729 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:38:07.607729 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:38:07.607729 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:38:07.607729 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:38:07.607729 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:38:07.609506 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 10:38:07.609506 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 10:38:07.609506 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 10:38:07.609506 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 10:38:07.609506 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 10:38:07.609506 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 10:38:07.609506 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 10:38:07.609506 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Jul 27 10:38:07.609506 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 622/1271] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Jul 27 10:38:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 10:38:07.614487 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c017a6-9434-4353-8b2b-344114abc6f1 None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:38:07.683251 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:38:07.724960 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c017a6-9434-4353-8b2b-344114abc6f1 None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:38:07.758327 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7662ae38-5111-439f-8003-88461606df46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:38:07.758978 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7662ae38-5111-439f-8003-88461606df46 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:38:07.759265 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7662ae38-5111-439f-8003-88461606df46 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:38:07.759540 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7662ae38-5111-439f-8003-88461606df46 None admin] PATH_INFO: `/v3/projects` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:38:07.760294 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7662ae38-5111-439f-8003-88461606df46 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86827) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:38:07.760605 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7662ae38-5111-439f-8003-88461606df46 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:38:07.762033 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7662ae38-5111-439f-8003-88461606df46 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:38:07.774271 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 650/1272] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Jul 27 10:38:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:38:07.780222 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3be56de-1343-4113-ada6-d2622b51114e None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:38:07.868909 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:38:07.878212 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3be56de-1343-4113-ada6-d2622b51114e None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:38:07.910898 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b8c0ca-ccb8-48db-960c-e144c6796d7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:38:07.911552 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b8c0ca-ccb8-48db-960c-e144c6796d7b None admin] REQUEST_METHOD: `PUT` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:38:07.911837 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b8c0ca-ccb8-48db-960c-e144c6796d7b None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:38:07.912117 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b8c0ca-ccb8-48db-960c-e144c6796d7b None admin] PATH_INFO: `/v3/projects/5f296d8e3b8f4001909addc49d8dac01/users/16cc6179e9a04b6398fbacb40ad1c555/roles/21156560eb6b436bb567a40b4976f3f4` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:38:07.914033 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b8c0ca-ccb8-48db-960c-e144c6796d7b None admin] RBAC: Authorizing `identity:create_grant(project_id=5f296d8e3b8f4001909addc49d8dac01, user_id=16cc6179e9a04b6398fbacb40ad1c555, role_id=21156560eb6b436bb567a40b4976f3f4)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:38:07.915490 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b8c0ca-ccb8-48db-960c-e144c6796d7b None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:38:07.928374 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 623/1273] 10.222.0.36 () {58 vars in 1439 bytes} [Wed Jul 27 10:38:07 2022] PUT /identity/v3/projects/5f296d8e3b8f4001909addc49d8dac01/users/16cc6179e9a04b6398fbacb40ad1c555/roles/21156560eb6b436bb567a40b4976f3f4 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 10:38:09.991183 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4562d5f-3e55-4ff8-94f1-894606f54d59 None None] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:38:09.991703 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4562d5f-3e55-4ff8-94f1-894606f54d59 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:38:09.991797 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4562d5f-3e55-4ff8-94f1-894606f54d59 None None] PATH_INFO: `/` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:38:09.993101 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 651/1274] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 10:38:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 10:38:09.998039 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529b6b97-a081-4715-9889-751965e3eeda None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:38:09.998453 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529b6b97-a081-4715-9889-751965e3eeda None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:38:09.998620 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529b6b97-a081-4715-9889-751965e3eeda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:38:10.017349 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-529b6b97-a081-4715-9889-751965e3eeda None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:38:10.018531 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-529b6b97-a081-4715-9889-751965e3eeda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:38:10.083523 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 624/1275] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 10:38:09 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 10:38:10.106795 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4562d5f-3e55-4ff8-94f1-894606f54d59 None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:38:10.208478 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4562d5f-3e55-4ff8-94f1-894606f54d59 None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:38:10.225473 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:38:10.243715 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9bdace-75a5-401f-a7ea-437f6d4aca1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:38:10.244343 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9bdace-75a5-401f-a7ea-437f6d4aca1a None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:38:10.244621 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9bdace-75a5-401f-a7ea-437f6d4aca1a None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:38:10.244892 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9bdace-75a5-401f-a7ea-437f6d4aca1a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:38:10.254684 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9bdace-75a5-401f-a7ea-437f6d4aca1a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 10:38:10.259781 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application [None req-3c9bdace-75a5-401f-a7ea-437f6d4aca1a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 10:38:10.259781 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:38:10.259781 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:38:10.259781 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:38:10.259781 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:38:10.259781 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:38:10.259781 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:38:10.259781 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:38:10.259781 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:38:10.259781 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:38:10.259781 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:38:10.259781 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:38:10.259781 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 10:38:10.259781 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 10:38:10.259781 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 10:38:10.259781 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 10:38:10.260875 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 10:38:10.260875 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 10:38:10.260875 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 10:38:10.260875 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 10:38:10.260875 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:38:10.260875 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:38:10.260875 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:38:10.260875 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:38:10.260875 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:38:10.260875 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:38:10.260875 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:38:10.260875 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application with Lock( Jul 27 10:38:10.260875 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:38:10.260875 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:38:10.260875 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:38:10.260875 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:38:10.261929 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:38:10.261929 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:38:10.261929 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:38:10.261929 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:38:10.261929 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 10:38:10.261929 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 10:38:10.261929 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 10:38:10.261929 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 10:38:10.261929 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 10:38:10.261929 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 10:38:10.261929 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 10:38:10.261929 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Jul 27 10:38:10.261929 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 652/1276] 10.222.0.36 () {56 vars in 1096 bytes} [Wed Jul 27 10:38:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 10:38:10.267076 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529b6b97-a081-4715-9889-751965e3eeda None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:38:10.345395 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:38:10.364549 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529b6b97-a081-4715-9889-751965e3eeda None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:38:10.397447 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d6a748-97c7-42e2-9e72-4def1d2a0e2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:38:10.398058 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d6a748-97c7-42e2-9e72-4def1d2a0e2c None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:38:10.398342 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d6a748-97c7-42e2-9e72-4def1d2a0e2c None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:38:10.398647 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d6a748-97c7-42e2-9e72-4def1d2a0e2c None admin] PATH_INFO: `/v3/roles` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:38:10.399361 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d6a748-97c7-42e2-9e72-4def1d2a0e2c None admin] RBAC: Adding query filter params (name=admin) {{(pid=86828) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:38:10.399668 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d6a748-97c7-42e2-9e72-4def1d2a0e2c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:38:10.401040 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d6a748-97c7-42e2-9e72-4def1d2a0e2c None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:38:10.409675 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 625/1277] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jul 27 10:38:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:38:10.415252 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9bdace-75a5-401f-a7ea-437f6d4aca1a None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:38:10.515038 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9bdace-75a5-401f-a7ea-437f6d4aca1a None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:38:10.548242 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312a5791-6431-4bf3-bfe6-ffea0e85e165 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:38:10.548900 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312a5791-6431-4bf3-bfe6-ffea0e85e165 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:38:10.549181 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312a5791-6431-4bf3-bfe6-ffea0e85e165 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:38:10.549452 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312a5791-6431-4bf3-bfe6-ffea0e85e165 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:38:10.559455 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312a5791-6431-4bf3-bfe6-ffea0e85e165 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:38:10.560951 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312a5791-6431-4bf3-bfe6-ffea0e85e165 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:38:10.575224 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application [None req-312a5791-6431-4bf3-bfe6-ffea0e85e165 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 10:38:10.575224 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:38:10.575224 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:38:10.575224 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:38:10.575224 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:38:10.575224 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:38:10.575224 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:38:10.575224 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:38:10.575224 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:38:10.575224 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:38:10.575224 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:38:10.575224 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:38:10.575224 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 10:38:10.575224 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 10:38:10.575224 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 10:38:10.576254 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 10:38:10.576254 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:38:10.576254 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:38:10.576254 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:38:10.576254 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:38:10.576254 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:38:10.576254 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:38:10.576254 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:38:10.576254 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application with Lock( Jul 27 10:38:10.576254 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:38:10.576254 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:38:10.576254 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:38:10.576254 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:38:10.576254 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:38:10.576254 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:38:10.576254 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:38:10.576254 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:38:10.577413 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 10:38:10.577413 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 10:38:10.577413 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 10:38:10.577413 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Jul 27 10:38:10.577413 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 653/1278] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 10:38:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 10:38:10.582752 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d6a748-97c7-42e2-9e72-4def1d2a0e2c None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:38:10.695546 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d6a748-97c7-42e2-9e72-4def1d2a0e2c None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:38:10.728944 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ffb8f7-befb-4f0b-96ce-bd4de94e24e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:38:10.729562 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ffb8f7-befb-4f0b-96ce-bd4de94e24e8 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:38:10.729844 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ffb8f7-befb-4f0b-96ce-bd4de94e24e8 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:38:10.730118 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ffb8f7-befb-4f0b-96ce-bd4de94e24e8 None admin] PATH_INFO: `/v3/domains` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:38:10.730831 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ffb8f7-befb-4f0b-96ce-bd4de94e24e8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86828) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:38:10.731136 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ffb8f7-befb-4f0b-96ce-bd4de94e24e8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:38:10.732500 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ffb8f7-befb-4f0b-96ce-bd4de94e24e8 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:38:10.743866 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 626/1279] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 10:38:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:38:10.749504 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312a5791-6431-4bf3-bfe6-ffea0e85e165 None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:38:10.779831 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:38:10.848280 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312a5791-6431-4bf3-bfe6-ffea0e85e165 None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:38:10.880730 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02d7f53-1e7d-4d51-9e6f-9a846e8bd859 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:38:10.881356 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02d7f53-1e7d-4d51-9e6f-9a846e8bd859 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:38:10.881641 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02d7f53-1e7d-4d51-9e6f-9a846e8bd859 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:38:10.881920 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02d7f53-1e7d-4d51-9e6f-9a846e8bd859 None admin] PATH_INFO: `/v3/users/placement` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:38:10.889451 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02d7f53-1e7d-4d51-9e6f-9a846e8bd859 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:38:10.890920 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02d7f53-1e7d-4d51-9e6f-9a846e8bd859 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:38:10.902544 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application [None req-f02d7f53-1e7d-4d51-9e6f-9a846e8bd859 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 27 10:38:10.902544 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:38:10.902544 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:38:10.902544 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:38:10.902544 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:38:10.902544 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:38:10.902544 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:38:10.902544 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:38:10.902544 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:38:10.902544 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:38:10.902544 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:38:10.902544 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:38:10.902544 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 10:38:10.902544 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 10:38:10.902544 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 10:38:10.902544 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 10:38:10.903637 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:38:10.903637 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:38:10.903637 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 10:38:10.903637 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 10:38:10.903637 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 10:38:10.903637 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 10:38:10.903637 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:38:10.903637 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:38:10.903637 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:38:10.903637 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:38:10.903637 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:38:10.903637 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application with Lock( Jul 27 10:38:10.903637 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:38:10.903637 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:38:10.903637 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:38:10.903637 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:38:10.903637 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:38:10.903637 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:38:10.904611 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:38:10.904611 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:38:10.904611 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 10:38:10.904611 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 10:38:10.904611 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 10:38:10.904611 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 10:38:10.904611 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 10:38:10.904611 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 10:38:10.904611 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 27 10:38:10.904611 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Jul 27 10:38:10.904611 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 654/1280] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 10:38:10 2022] GET /identity/v3/users/placement => generated 87 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 10:38:10.909636 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ffb8f7-befb-4f0b-96ce-bd4de94e24e8 None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:38:10.915079 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:38:11.012334 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ffb8f7-befb-4f0b-96ce-bd4de94e24e8 None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:38:11.045331 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec647f0-1380-407e-b0f5-868f6ce7a2c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:38:11.045885 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec647f0-1380-407e-b0f5-868f6ce7a2c0 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:38:11.046171 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec647f0-1380-407e-b0f5-868f6ce7a2c0 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:38:11.046450 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec647f0-1380-407e-b0f5-868f6ce7a2c0 None admin] PATH_INFO: `/v3/users` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:38:11.047272 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec647f0-1380-407e-b0f5-868f6ce7a2c0 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=86828) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:38:11.047585 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec647f0-1380-407e-b0f5-868f6ce7a2c0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:38:11.049038 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec647f0-1380-407e-b0f5-868f6ce7a2c0 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:38:11.066710 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 627/1281] 10.222.0.36 () {56 vars in 1143 bytes} [Wed Jul 27 10:38:10 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:38:11.072621 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02d7f53-1e7d-4d51-9e6f-9a846e8bd859 None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:38:11.162183 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:38:11.171897 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02d7f53-1e7d-4d51-9e6f-9a846e8bd859 None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:38:11.204961 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c982d78-225f-4d8b-9a26-9b537868e543 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:38:11.205596 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c982d78-225f-4d8b-9a26-9b537868e543 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:38:11.205886 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c982d78-225f-4d8b-9a26-9b537868e543 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:38:11.206163 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c982d78-225f-4d8b-9a26-9b537868e543 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:38:11.280403 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c982d78-225f-4d8b-9a26-9b537868e543 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:38:11.281852 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c982d78-225f-4d8b-9a26-9b537868e543 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:38:11.367239 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:38:11.378174 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 655/1282] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jul 27 10:38:11 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 27 10:38:11.384312 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec647f0-1380-407e-b0f5-868f6ce7a2c0 None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:38:11.443967 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:38:11.481405 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec647f0-1380-407e-b0f5-868f6ce7a2c0 None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:38:11.515377 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4376d5dd-d8ee-49c7-9aa3-dae6691113c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:38:11.516045 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4376d5dd-d8ee-49c7-9aa3-dae6691113c1 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:38:11.516325 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4376d5dd-d8ee-49c7-9aa3-dae6691113c1 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:38:11.516598 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4376d5dd-d8ee-49c7-9aa3-dae6691113c1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:38:11.526434 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4376d5dd-d8ee-49c7-9aa3-dae6691113c1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:38:11.527895 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4376d5dd-d8ee-49c7-9aa3-dae6691113c1 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:38:11.543193 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application [None req-4376d5dd-d8ee-49c7-9aa3-dae6691113c1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 10:38:11.543193 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:38:11.543193 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:38:11.543193 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:38:11.543193 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:38:11.543193 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:38:11.543193 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:38:11.543193 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:38:11.543193 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:38:11.543193 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:38:11.543193 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:38:11.543193 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:38:11.543193 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 10:38:11.543193 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 10:38:11.543193 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 10:38:11.544255 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 10:38:11.544255 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:38:11.544255 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:38:11.544255 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:38:11.544255 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:38:11.544255 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:38:11.544255 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:38:11.544255 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:38:11.544255 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application with Lock( Jul 27 10:38:11.544255 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:38:11.544255 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:38:11.544255 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:38:11.544255 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:38:11.544255 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:38:11.544255 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:38:11.544255 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:38:11.544255 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:38:11.545213 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 10:38:11.545213 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 10:38:11.545213 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 10:38:11.545213 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Jul 27 10:38:11.545213 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 628/1283] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 10:38:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 10:38:11.550836 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c982d78-225f-4d8b-9a26-9b537868e543 None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:38:11.651518 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c982d78-225f-4d8b-9a26-9b537868e543 None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:38:11.685089 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89892e15-8c5c-4e15-a43f-a3d595e52105 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:38:11.685708 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89892e15-8c5c-4e15-a43f-a3d595e52105 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:38:11.685991 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89892e15-8c5c-4e15-a43f-a3d595e52105 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:38:11.686268 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89892e15-8c5c-4e15-a43f-a3d595e52105 None admin] PATH_INFO: `/v3/domains` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:38:11.686988 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89892e15-8c5c-4e15-a43f-a3d595e52105 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86827) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:38:11.687296 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89892e15-8c5c-4e15-a43f-a3d595e52105 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:38:11.688671 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89892e15-8c5c-4e15-a43f-a3d595e52105 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:38:11.700062 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 656/1284] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 10:38:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:38:11.706280 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4376d5dd-d8ee-49c7-9aa3-dae6691113c1 None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:38:11.802891 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4376d5dd-d8ee-49c7-9aa3-dae6691113c1 None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:38:11.816243 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:38:11.836316 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd201f2-08ab-48dd-b930-76cf80c3e995 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:38:11.836938 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd201f2-08ab-48dd-b930-76cf80c3e995 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:38:11.837216 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd201f2-08ab-48dd-b930-76cf80c3e995 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:38:11.837484 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd201f2-08ab-48dd-b930-76cf80c3e995 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:38:11.843880 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd201f2-08ab-48dd-b930-76cf80c3e995 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:38:11.845309 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd201f2-08ab-48dd-b930-76cf80c3e995 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:38:11.856271 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application [None req-9fd201f2-08ab-48dd-b930-76cf80c3e995 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 10:38:11.856271 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:38:11.856271 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:38:11.856271 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:38:11.856271 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:38:11.856271 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:38:11.856271 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:38:11.856271 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:38:11.856271 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:38:11.856271 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:38:11.856271 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:38:11.856271 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:38:11.856271 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 10:38:11.856271 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 10:38:11.856271 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 10:38:11.857314 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 10:38:11.857314 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:38:11.857314 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:38:11.857314 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:38:11.857314 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:38:11.857314 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:38:11.857314 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:38:11.857314 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:38:11.857314 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application with Lock( Jul 27 10:38:11.857314 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:38:11.857314 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:38:11.857314 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:38:11.857314 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:38:11.857314 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:38:11.857314 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:38:11.857314 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:38:11.857314 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:38:11.858421 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 10:38:11.858421 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 10:38:11.858421 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 10:38:11.858421 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 10:38:11.858421 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 10:38:11.858421 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 10:38:11.858421 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 10:38:11.858421 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: ERROR keystone.server.flask.application Jul 27 10:38:11.858421 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 629/1285] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Jul 27 10:38:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 10:38:11.863135 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89892e15-8c5c-4e15-a43f-a3d595e52105 None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:38:11.913354 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:38:11.962793 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89892e15-8c5c-4e15-a43f-a3d595e52105 None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:38:11.995315 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68021f1d-0e4b-4174-8f0c-2c037e7784c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:38:11.995928 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68021f1d-0e4b-4174-8f0c-2c037e7784c4 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:38:11.996215 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68021f1d-0e4b-4174-8f0c-2c037e7784c4 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:38:11.996492 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68021f1d-0e4b-4174-8f0c-2c037e7784c4 None admin] PATH_INFO: `/v3/projects` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:38:11.997243 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68021f1d-0e4b-4174-8f0c-2c037e7784c4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86827) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:38:11.997563 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68021f1d-0e4b-4174-8f0c-2c037e7784c4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:38:11.999019 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68021f1d-0e4b-4174-8f0c-2c037e7784c4 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:38:12.011407 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 657/1286] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Jul 27 10:38:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:38:12.017463 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd201f2-08ab-48dd-b930-76cf80c3e995 None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:38:12.114267 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd201f2-08ab-48dd-b930-76cf80c3e995 None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:38:12.147372 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5755dc22-90e6-4856-a35f-496f73c7cf3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:38:12.147983 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5755dc22-90e6-4856-a35f-496f73c7cf3c None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:38:12.148271 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5755dc22-90e6-4856-a35f-496f73c7cf3c None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:38:12.148550 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5755dc22-90e6-4856-a35f-496f73c7cf3c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:38:12.149336 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5755dc22-90e6-4856-a35f-496f73c7cf3c None admin] RBAC: Adding query filter params (role.id=21156560eb6b436bb567a40b4976f3f4, scope.project.id=5f296d8e3b8f4001909addc49d8dac01, user.id=16cc6179e9a04b6398fbacb40ad1c555) {{(pid=86828) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:38:12.149648 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5755dc22-90e6-4856-a35f-496f73c7cf3c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:38:12.151179 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5755dc22-90e6-4856-a35f-496f73c7cf3c None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:38:12.156859 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 630/1287] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Jul 27 10:38:12 2022] GET /identity/v3/role_assignments?user.id=16cc6179e9a04b6398fbacb40ad1c555&scope.project.id=5f296d8e3b8f4001909addc49d8dac01&role.id=21156560eb6b436bb567a40b4976f3f4 => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:38:14.155332 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ebeb17-f36d-4a95-9c15-cd5195e45bfa None None] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:38:14.155877 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ebeb17-f36d-4a95-9c15-cd5195e45bfa None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:38:14.155877 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ebeb17-f36d-4a95-9c15-cd5195e45bfa None None] PATH_INFO: `/` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:38:14.157255 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 658/1288] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 10:38:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 10:38:14.162239 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70d170b-91bf-4101-8e80-4d163940a54a None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:38:14.162698 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70d170b-91bf-4101-8e80-4d163940a54a None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:38:14.162830 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70d170b-91bf-4101-8e80-4d163940a54a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:38:14.181036 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-f70d170b-91bf-4101-8e80-4d163940a54a None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:38:14.182206 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-f70d170b-91bf-4101-8e80-4d163940a54a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:38:14.211474 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:38:14.247901 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 631/1289] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 10:38:14 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 10:38:14.270132 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ebeb17-f36d-4a95-9c15-cd5195e45bfa None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:38:14.368497 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ebeb17-f36d-4a95-9c15-cd5195e45bfa None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:38:14.379325 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:38:14.402344 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e518b731-e146-4010-b84b-20a7171d6743 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:38:14.403010 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e518b731-e146-4010-b84b-20a7171d6743 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:38:14.403299 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e518b731-e146-4010-b84b-20a7171d6743 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:38:14.403580 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e518b731-e146-4010-b84b-20a7171d6743 None admin] PATH_INFO: `/v3/services/placement` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:38:14.404208 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e518b731-e146-4010-b84b-20a7171d6743 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:38:14.405579 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e518b731-e146-4010-b84b-20a7171d6743 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:38:14.414297 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application [None req-e518b731-e146-4010-b84b-20a7171d6743 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 27 10:38:14.414297 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:38:14.414297 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 27 10:38:14.414297 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 27 10:38:14.414297 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 27 10:38:14.414297 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 27 10:38:14.414297 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 27 10:38:14.414297 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 10:38:14.414297 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 27 10:38:14.414297 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Jul 27 10:38:14.414297 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 10:38:14.414297 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Jul 27 10:38:14.414297 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:38:14.414297 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:38:14.414297 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:38:14.414297 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:38:14.414297 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:38:14.415532 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:38:14.415532 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:38:14.415532 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:38:14.415532 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:38:14.415532 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:38:14.415532 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:38:14.415532 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 27 10:38:14.415532 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 27 10:38:14.415532 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 27 10:38:14.415532 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 27 10:38:14.415532 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:38:14.415532 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:38:14.415532 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:38:14.415532 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:38:14.415532 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:38:14.415532 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:38:14.416579 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:38:14.416579 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application with Lock( Jul 27 10:38:14.416579 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:38:14.416579 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:38:14.416579 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:38:14.416579 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:38:14.416579 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:38:14.416579 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:38:14.416579 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:38:14.416579 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:38:14.416579 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 27 10:38:14.416579 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 10:38:14.416579 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 27 10:38:14.416579 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Jul 27 10:38:14.416579 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 659/1290] 10.222.0.36 () {56 vars in 1117 bytes} [Wed Jul 27 10:38:14 2022] GET /identity/v3/services/placement => generated 90 bytes in 146 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 10:38:14.421354 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70d170b-91bf-4101-8e80-4d163940a54a None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:38:14.520618 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70d170b-91bf-4101-8e80-4d163940a54a None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:38:14.553597 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d05787-b1a7-4a64-8e1d-41e45e6b2e97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:38:14.554204 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d05787-b1a7-4a64-8e1d-41e45e6b2e97 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:38:14.554480 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d05787-b1a7-4a64-8e1d-41e45e6b2e97 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:38:14.554780 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d05787-b1a7-4a64-8e1d-41e45e6b2e97 None admin] PATH_INFO: `/v3/services` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:38:14.555449 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d05787-b1a7-4a64-8e1d-41e45e6b2e97 None admin] RBAC: Adding query filter params (name=placement) {{(pid=86828) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:38:14.555748 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d05787-b1a7-4a64-8e1d-41e45e6b2e97 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:38:14.557097 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d05787-b1a7-4a64-8e1d-41e45e6b2e97 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:38:14.561732 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 632/1291] 10.222.0.36 () {56 vars in 1116 bytes} [Wed Jul 27 10:38:14 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:38:14.567965 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e518b731-e146-4010-b84b-20a7171d6743 None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:38:14.683716 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e518b731-e146-4010-b84b-20a7171d6743 None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:38:14.716608 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81e6e35-c81b-4fc5-8294-142128b28f58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:38:14.717223 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81e6e35-c81b-4fc5-8294-142128b28f58 None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:38:14.717499 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81e6e35-c81b-4fc5-8294-142128b28f58 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:38:14.717769 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81e6e35-c81b-4fc5-8294-142128b28f58 None admin] PATH_INFO: `/v3/services` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:38:14.718438 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81e6e35-c81b-4fc5-8294-142128b28f58 None admin] RBAC: Adding query filter params (type=placement) {{(pid=86827) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:38:14.718768 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81e6e35-c81b-4fc5-8294-142128b28f58 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:38:14.720125 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81e6e35-c81b-4fc5-8294-142128b28f58 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:38:14.724424 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 660/1292] 10.222.0.36 () {56 vars in 1116 bytes} [Wed Jul 27 10:38:14 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:38:16.690575 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f209b4-ba2e-4333-a471-b9d23208bc79 None None] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:38:16.691185 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f209b4-ba2e-4333-a471-b9d23208bc79 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:38:16.691185 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f209b4-ba2e-4333-a471-b9d23208bc79 None None] PATH_INFO: `/` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:38:16.692504 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 633/1293] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 10:38:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 10:38:16.697477 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba748c4d-328d-4a4e-912c-0583d51d8971 None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:38:16.697927 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba748c4d-328d-4a4e-912c-0583d51d8971 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:38:16.698030 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba748c4d-328d-4a4e-912c-0583d51d8971 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:38:16.716277 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-ba748c4d-328d-4a4e-912c-0583d51d8971 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:38:16.717464 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-ba748c4d-328d-4a4e-912c-0583d51d8971 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:38:16.746616 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:38:16.784628 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 661/1294] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 10:38:16 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 10:38:16.806698 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f209b4-ba2e-4333-a471-b9d23208bc79 None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:38:16.837563 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:38:16.905123 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f209b4-ba2e-4333-a471-b9d23208bc79 None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:38:16.938191 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13debd56-f39e-4394-abfe-1eabdc1cd417 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:38:16.938837 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13debd56-f39e-4394-abfe-1eabdc1cd417 None admin] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:38:16.939119 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13debd56-f39e-4394-abfe-1eabdc1cd417 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:38:16.939915 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13debd56-f39e-4394-abfe-1eabdc1cd417 None admin] PATH_INFO: `/v3/services` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:38:16.940671 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13debd56-f39e-4394-abfe-1eabdc1cd417 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:38:16.942051 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13debd56-f39e-4394-abfe-1eabdc1cd417 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:38:16.956732 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 634/1295] 10.222.0.36 () {60 vars in 1141 bytes} [Wed Jul 27 10:38:16 2022] POST /identity/v3/services => generated 246 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 10:38:18.964955 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7b99f6-ee2a-41e0-b956-61e250344795 None None] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:38:18.965425 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7b99f6-ee2a-41e0-b956-61e250344795 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:38:18.965526 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7b99f6-ee2a-41e0-b956-61e250344795 None None] PATH_INFO: `/` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:38:18.966907 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 662/1296] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 10:38:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 10:38:18.971989 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a42048-8112-476f-bc79-ee5e02d36ff0 None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:38:18.972388 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a42048-8112-476f-bc79-ee5e02d36ff0 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:38:18.972541 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a42048-8112-476f-bc79-ee5e02d36ff0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:38:18.990687 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-c7a42048-8112-476f-bc79-ee5e02d36ff0 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:38:18.991860 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-c7a42048-8112-476f-bc79-ee5e02d36ff0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:38:19.068445 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 635/1297] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 10:38:18 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 10:38:19.090464 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7b99f6-ee2a-41e0-b956-61e250344795 None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:38:19.189665 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7b99f6-ee2a-41e0-b956-61e250344795 None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:38:19.222505 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2f2ef3-131b-4535-95df-573e2e353fbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:38:19.223095 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2f2ef3-131b-4535-95df-573e2e353fbe None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:38:19.223380 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2f2ef3-131b-4535-95df-573e2e353fbe None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:38:19.223655 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2f2ef3-131b-4535-95df-573e2e353fbe None admin] PATH_INFO: `/v3/services/placement` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:38:19.224277 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2f2ef3-131b-4535-95df-573e2e353fbe None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:38:19.225640 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2f2ef3-131b-4535-95df-573e2e353fbe None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:38:19.234342 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application [None req-6d2f2ef3-131b-4535-95df-573e2e353fbe None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 27 10:38:19.234342 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:38:19.234342 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 27 10:38:19.234342 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 27 10:38:19.234342 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 27 10:38:19.234342 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 27 10:38:19.234342 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 27 10:38:19.234342 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 10:38:19.234342 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 27 10:38:19.234342 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Jul 27 10:38:19.234342 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 10:38:19.234342 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Jul 27 10:38:19.234342 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:38:19.234342 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:38:19.234342 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:38:19.234342 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:38:19.234342 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:38:19.235659 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:38:19.235659 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:38:19.235659 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:38:19.235659 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:38:19.235659 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:38:19.235659 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:38:19.235659 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 27 10:38:19.235659 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 27 10:38:19.235659 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 27 10:38:19.235659 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 27 10:38:19.235659 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:38:19.235659 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:38:19.235659 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:38:19.235659 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:38:19.235659 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:38:19.235659 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:38:19.236722 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:38:19.236722 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application with Lock( Jul 27 10:38:19.236722 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:38:19.236722 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:38:19.236722 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:38:19.236722 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:38:19.236722 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:38:19.236722 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:38:19.236722 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:38:19.236722 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:38:19.236722 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 27 10:38:19.236722 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 10:38:19.236722 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 27 10:38:19.236722 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Jul 27 10:38:19.236722 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 663/1298] 10.222.0.36 () {56 vars in 1117 bytes} [Wed Jul 27 10:38:19 2022] GET /identity/v3/services/placement => generated 90 bytes in 146 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 10:38:19.241639 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a42048-8112-476f-bc79-ee5e02d36ff0 None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:38:19.260190 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:38:19.342079 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a42048-8112-476f-bc79-ee5e02d36ff0 None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:38:19.374391 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa6d497-3152-48a0-94ab-769ebeb06fad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:38:19.375035 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa6d497-3152-48a0-94ab-769ebeb06fad None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:38:19.375321 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa6d497-3152-48a0-94ab-769ebeb06fad None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:38:19.375600 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa6d497-3152-48a0-94ab-769ebeb06fad None admin] PATH_INFO: `/v3/services` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:38:19.376273 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa6d497-3152-48a0-94ab-769ebeb06fad None admin] RBAC: Adding query filter params (name=placement) {{(pid=86828) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:38:19.376580 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa6d497-3152-48a0-94ab-769ebeb06fad None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:38:19.378456 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa6d497-3152-48a0-94ab-769ebeb06fad None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:38:19.383729 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 636/1299] 10.222.0.36 () {56 vars in 1116 bytes} [Wed Jul 27 10:38:19 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:38:19.389653 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2f2ef3-131b-4535-95df-573e2e353fbe None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:38:19.420871 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:38:19.488579 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2f2ef3-131b-4535-95df-573e2e353fbe None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:38:19.521630 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96cff56-cca8-4804-9806-bf1b76c9905f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:38:19.522236 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96cff56-cca8-4804-9806-bf1b76c9905f None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:38:19.522517 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96cff56-cca8-4804-9806-bf1b76c9905f None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:38:19.522821 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96cff56-cca8-4804-9806-bf1b76c9905f None admin] PATH_INFO: `/v3/endpoints` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:38:19.523569 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96cff56-cca8-4804-9806-bf1b76c9905f None admin] RBAC: Adding query filter params (interface=public, service_id=ba0e28dfde78413198e84efefda14a52, region_id=RegionOne) {{(pid=86827) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:38:19.523878 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96cff56-cca8-4804-9806-bf1b76c9905f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:38:19.525228 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96cff56-cca8-4804-9806-bf1b76c9905f None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:38:19.530132 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 664/1300] 10.222.0.36 () {56 vars in 1251 bytes} [Wed Jul 27 10:38:19 2022] GET /identity/v3/endpoints?service_id=ba0e28dfde78413198e84efefda14a52&interface=public®ion_id=RegionOne => generated 194 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:38:19.535625 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa6d497-3152-48a0-94ab-769ebeb06fad None admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:38:19.614523 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:38:19.632780 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa6d497-3152-48a0-94ab-769ebeb06fad None admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:38:19.664925 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301b8338-c0a8-471f-80db-60b7ac20ede6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:38:19.665536 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301b8338-c0a8-471f-80db-60b7ac20ede6 None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:38:19.665819 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301b8338-c0a8-471f-80db-60b7ac20ede6 None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:38:19.666094 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301b8338-c0a8-471f-80db-60b7ac20ede6 None admin] PATH_INFO: `/v3/services` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:38:19.666769 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301b8338-c0a8-471f-80db-60b7ac20ede6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:38:19.668126 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301b8338-c0a8-471f-80db-60b7ac20ede6 None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:38:19.675247 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 637/1301] 10.222.0.36 () {56 vars in 1087 bytes} [Wed Jul 27 10:38:19 2022] GET /identity/v3/services => generated 2191 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 10:38:21.653315 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d41cc51-5ebc-49a3-b8ba-7fe3d5bc37f1 None None] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:38:21.653891 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d41cc51-5ebc-49a3-b8ba-7fe3d5bc37f1 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:38:21.653891 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d41cc51-5ebc-49a3-b8ba-7fe3d5bc37f1 None None] PATH_INFO: `/` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:38:21.655275 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 665/1302] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 10:38:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 10:38:21.660347 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef4a13f-d646-4a68-b090-6cdcfcf59039 None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:38:21.660718 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef4a13f-d646-4a68-b090-6cdcfcf59039 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:38:21.660892 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef4a13f-d646-4a68-b090-6cdcfcf59039 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:38:21.678913 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-eef4a13f-d646-4a68-b090-6cdcfcf59039 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:38:21.680078 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-eef4a13f-d646-4a68-b090-6cdcfcf59039 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:38:21.743450 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 638/1303] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 10:38:21 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 10:38:21.766229 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d41cc51-5ebc-49a3-b8ba-7fe3d5bc37f1 None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:38:21.858785 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:38:21.866369 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d41cc51-5ebc-49a3-b8ba-7fe3d5bc37f1 None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:38:21.901108 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e69a42-7173-4be7-9f19-eabcba85c89d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:38:21.901741 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e69a42-7173-4be7-9f19-eabcba85c89d None admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:38:21.902021 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e69a42-7173-4be7-9f19-eabcba85c89d None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:38:21.902295 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e69a42-7173-4be7-9f19-eabcba85c89d None admin] PATH_INFO: `/v3/services/placement` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:38:21.902952 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e69a42-7173-4be7-9f19-eabcba85c89d None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:38:21.904428 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e69a42-7173-4be7-9f19-eabcba85c89d None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:38:21.913102 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application [None req-64e69a42-7173-4be7-9f19-eabcba85c89d None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 27 10:38:21.913102 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:38:21.913102 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 27 10:38:21.913102 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 27 10:38:21.913102 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 27 10:38:21.913102 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 27 10:38:21.913102 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 27 10:38:21.913102 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 10:38:21.913102 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 27 10:38:21.913102 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Jul 27 10:38:21.913102 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 10:38:21.913102 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Jul 27 10:38:21.913102 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 10:38:21.913102 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 10:38:21.913102 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 10:38:21.913102 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 10:38:21.913102 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 10:38:21.914250 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 10:38:21.914250 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 10:38:21.914250 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 10:38:21.914250 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 10:38:21.914250 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 10:38:21.914250 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 10:38:21.914250 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 27 10:38:21.914250 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 27 10:38:21.914250 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 27 10:38:21.914250 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 27 10:38:21.914250 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 10:38:21.914250 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 10:38:21.914250 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 10:38:21.914250 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 10:38:21.914250 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 10:38:21.914250 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 10:38:21.915179 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 10:38:21.915179 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application with Lock( Jul 27 10:38:21.915179 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 10:38:21.915179 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self._enter() Jul 27 10:38:21.915179 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 10:38:21.915179 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 10:38:21.915179 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 10:38:21.915179 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application return self.creator() Jul 27 10:38:21.915179 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 10:38:21.915179 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application created_value = creator( Jul 27 10:38:21.915179 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 27 10:38:21.915179 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 10:38:21.915179 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 27 10:38:21.915179 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: ERROR keystone.server.flask.application Jul 27 10:38:21.915179 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 666/1304] 10.222.0.36 () {56 vars in 1117 bytes} [Wed Jul 27 10:38:21 2022] GET /identity/v3/services/placement => generated 90 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 10:38:21.920266 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef4a13f-d646-4a68-b090-6cdcfcf59039 None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:38:21.993392 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:38:22.016147 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef4a13f-d646-4a68-b090-6cdcfcf59039 None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:38:22.050035 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed2cd2f-dbc8-4136-9725-c87f7d41b48f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:38:22.050675 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed2cd2f-dbc8-4136-9725-c87f7d41b48f None admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:38:22.050961 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed2cd2f-dbc8-4136-9725-c87f7d41b48f None admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:38:22.051237 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed2cd2f-dbc8-4136-9725-c87f7d41b48f None admin] PATH_INFO: `/v3/services` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:38:22.051909 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed2cd2f-dbc8-4136-9725-c87f7d41b48f None admin] RBAC: Adding query filter params (name=placement) {{(pid=86828) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:38:22.052215 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed2cd2f-dbc8-4136-9725-c87f7d41b48f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:38:22.053577 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed2cd2f-dbc8-4136-9725-c87f7d41b48f None admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:38:22.058544 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 639/1305] 10.222.0.36 () {56 vars in 1116 bytes} [Wed Jul 27 10:38:21 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:38:22.064384 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e69a42-7173-4be7-9f19-eabcba85c89d None admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:38:22.159993 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e69a42-7173-4be7-9f19-eabcba85c89d None admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:38:22.192777 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1814185-1c58-48ba-a9f3-bad48bc7c3c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:38:22.193371 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1814185-1c58-48ba-a9f3-bad48bc7c3c2 None admin] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:38:22.193650 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1814185-1c58-48ba-a9f3-bad48bc7c3c2 None admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:38:22.193920 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1814185-1c58-48ba-a9f3-bad48bc7c3c2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:38:22.194631 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1814185-1c58-48ba-a9f3-bad48bc7c3c2 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:38:22.195987 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1814185-1c58-48ba-a9f3-bad48bc7c3c2 None admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:38:22.214860 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 667/1306] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Jul 27 10:38:22 2022] POST /identity/v3/endpoints => generated 331 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 10:38:32.055637 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=30 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:38:32.210800 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:39:25.229658 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a65a92-90f2-49d9-9da6-71fc994c4869 None None] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:39:25.230140 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a65a92-90f2-49d9-9da6-71fc994c4869 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:39:25.230485 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a65a92-90f2-49d9-9da6-71fc994c4869 None None] PATH_INFO: `/` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:39:25.235874 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 640/1307] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 10:39:25 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 10:39:25.237276 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d80844-6342-4a02-a7b6-398bacd862a0 None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:39:25.237724 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d80844-6342-4a02-a7b6-398bacd862a0 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:39:25.237890 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d80844-6342-4a02-a7b6-398bacd862a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:39:25.321389 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-86d80844-6342-4a02-a7b6-398bacd862a0 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:39:25.322643 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-86d80844-6342-4a02-a7b6-398bacd862a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:39:25.447669 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 668/1308] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 10:39:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 10:39:25.497946 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36c7c7c-2e2c-42dd-b5d9-6478ff2afee9 None None] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:39:25.498229 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36c7c7c-2e2c-42dd-b5d9-6478ff2afee9 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:39:25.498692 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36c7c7c-2e2c-42dd-b5d9-6478ff2afee9 None None] PATH_INFO: `/` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:39:25.500580 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 641/1309] 10.222.0.36 () {54 vars in 965 bytes} [Wed Jul 27 10:39:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 10:39:25.513854 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aee897d-e758-4428-9ea5-d4376352b253 None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:39:25.514198 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aee897d-e758-4428-9ea5-d4376352b253 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:39:25.514943 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aee897d-e758-4428-9ea5-d4376352b253 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:39:25.614750 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-9aee897d-e758-4428-9ea5-d4376352b253 None None] MFA Rules not processed for user `c7922d0a1a7c4e808d0b2a80dca0da40`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:39:25.616439 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-9aee897d-e758-4428-9ea5-d4376352b253 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:39:25.640541 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:39:25.703075 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 669/1310] 10.222.0.36 () {58 vars in 1062 bytes} [Wed Jul 27 10:39:25 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:39:25.711825 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36c7c7c-2e2c-42dd-b5d9-6478ff2afee9 None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:39:25.713845 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-f36c7c7c-2e2c-42dd-b5d9-6478ff2afee9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:39:25.834548 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36c7c7c-2e2c-42dd-b5d9-6478ff2afee9 None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:39:25.915565 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe3371b-734f-41fd-b449-25ea4b3baeef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7922d0a1a7c4e808d0b2a80dca0da40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:39:25.916236 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe3371b-734f-41fd-b449-25ea4b3baeef service neutron] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:39:25.916531 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe3371b-734f-41fd-b449-25ea4b3baeef service neutron] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:39:25.916939 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe3371b-734f-41fd-b449-25ea4b3baeef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:39:25.918575 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-ffe3371b-734f-41fd-b449-25ea4b3baeef service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:39:25.977602 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:39:26.039917 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe3371b-734f-41fd-b449-25ea4b3baeef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:39:26.041366 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe3371b-734f-41fd-b449-25ea4b3baeef service neutron] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:39:26.068148 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 642/1311] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:39:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 10:39:28.435398 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c26f33-ba65-4d03-ab14-3dd6d462f4c3 None None] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:39:28.435936 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c26f33-ba65-4d03-ab14-3dd6d462f4c3 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:39:28.435936 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c26f33-ba65-4d03-ab14-3dd6d462f4c3 None None] PATH_INFO: `/` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:39:28.437369 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 670/1312] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 10:39:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 10:39:28.443267 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d3cb18-801a-459a-b613-3be2ee0bb2ba None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:39:28.443709 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d3cb18-801a-459a-b613-3be2ee0bb2ba None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:39:28.443810 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d3cb18-801a-459a-b613-3be2ee0bb2ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:39:28.526250 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-d6d3cb18-801a-459a-b613-3be2ee0bb2ba None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:39:28.527555 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-d6d3cb18-801a-459a-b613-3be2ee0bb2ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:39:28.627853 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 643/1313] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 10:39:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:39:28.666410 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c26f33-ba65-4d03-ab14-3dd6d462f4c3 None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:39:28.668018 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-69c26f33-ba65-4d03-ab14-3dd6d462f4c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:39:28.773829 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c26f33-ba65-4d03-ab14-3dd6d462f4c3 None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:39:28.854510 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:39:28.857583 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1004b3-9ee8-4a92-b3d8-fd63e6ae8058 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7922d0a1a7c4e808d0b2a80dca0da40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:39:28.858415 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1004b3-9ee8-4a92-b3d8-fd63e6ae8058 service neutron] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:39:28.858910 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1004b3-9ee8-4a92-b3d8-fd63e6ae8058 service neutron] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:39:28.860077 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1004b3-9ee8-4a92-b3d8-fd63e6ae8058 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:39:28.861804 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-0e1004b3-9ee8-4a92-b3d8-fd63e6ae8058 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:39:28.985075 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1004b3-9ee8-4a92-b3d8-fd63e6ae8058 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:39:28.986525 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1004b3-9ee8-4a92-b3d8-fd63e6ae8058 service neutron] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:39:29.004411 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 671/1314] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:39:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 10:39:31.326993 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266329f6-6651-4d9d-a0c6-175feaad33cd None None] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:39:31.327504 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266329f6-6651-4d9d-a0c6-175feaad33cd None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:39:31.327598 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266329f6-6651-4d9d-a0c6-175feaad33cd None None] PATH_INFO: `/` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:39:31.328943 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 644/1315] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 10:39:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 10:39:31.333936 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d6551c-bcdd-40df-b1cb-a444d8b8ff0d None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:39:31.334379 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d6551c-bcdd-40df-b1cb-a444d8b8ff0d None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:39:31.334501 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d6551c-bcdd-40df-b1cb-a444d8b8ff0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:39:31.404885 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-a3d6551c-bcdd-40df-b1cb-a444d8b8ff0d None None] MFA Rules not processed for user `6832ba6fe7d34e438a50d7f4d5681d00`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:39:31.406096 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-a3d6551c-bcdd-40df-b1cb-a444d8b8ff0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:39:31.457239 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:39:31.502391 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 672/1316] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 10:39:31 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:39:31.542703 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266329f6-6651-4d9d-a0c6-175feaad33cd None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:39:31.544229 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-266329f6-6651-4d9d-a0c6-175feaad33cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:39:31.554571 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:39:31.648451 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266329f6-6651-4d9d-a0c6-175feaad33cd None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:39:31.720141 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245cd148-ce3b-417f-b313-758c7cf77990 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7922d0a1a7c4e808d0b2a80dca0da40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:39:31.720759 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245cd148-ce3b-417f-b313-758c7cf77990 service neutron] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:39:31.721045 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245cd148-ce3b-417f-b313-758c7cf77990 service neutron] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:39:31.721324 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245cd148-ce3b-417f-b313-758c7cf77990 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:39:31.722989 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-245cd148-ce3b-417f-b313-758c7cf77990 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:39:31.784642 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:39:31.831588 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245cd148-ce3b-417f-b313-758c7cf77990 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:39:31.833021 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245cd148-ce3b-417f-b313-758c7cf77990 service neutron] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:39:31.851017 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 645/1317] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:39:31 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 10:39:34.405841 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73e0daa-7730-4c73-a39d-5f51c2444654 None None] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:39:34.407019 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73e0daa-7730-4c73-a39d-5f51c2444654 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:39:34.407149 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73e0daa-7730-4c73-a39d-5f51c2444654 None None] PATH_INFO: `/` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:39:34.408578 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 673/1318] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 10:39:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 10:39:34.413918 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a644bb12-489a-484c-825b-86e7ccdf608d None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:39:34.414319 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a644bb12-489a-484c-825b-86e7ccdf608d None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:39:34.414465 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a644bb12-489a-484c-825b-86e7ccdf608d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:39:34.478454 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-a644bb12-489a-484c-825b-86e7ccdf608d None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:39:34.479671 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-a644bb12-489a-484c-825b-86e7ccdf608d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:39:34.575581 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 646/1319] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 10:39:34 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:39:34.615342 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73e0daa-7730-4c73-a39d-5f51c2444654 None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:39:34.617555 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-b73e0daa-7730-4c73-a39d-5f51c2444654 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:39:34.719907 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73e0daa-7730-4c73-a39d-5f51c2444654 None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:39:34.791580 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129f7a56-1c61-4db2-bd50-a2c8468eb5b2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7922d0a1a7c4e808d0b2a80dca0da40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:39:34.792193 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129f7a56-1c61-4db2-bd50-a2c8468eb5b2 service neutron] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:39:34.792475 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129f7a56-1c61-4db2-bd50-a2c8468eb5b2 service neutron] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:39:34.792752 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129f7a56-1c61-4db2-bd50-a2c8468eb5b2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:39:34.794373 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-129f7a56-1c61-4db2-bd50-a2c8468eb5b2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:39:34.802287 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:39:34.904258 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129f7a56-1c61-4db2-bd50-a2c8468eb5b2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:39:34.905695 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129f7a56-1c61-4db2-bd50-a2c8468eb5b2 service neutron] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:39:34.923927 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 674/1320] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:39:34 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 10:39:37.572707 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcd3317-a2d7-4228-bdb0-e4ffc7efc7d2 None None] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:39:37.573259 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcd3317-a2d7-4228-bdb0-e4ffc7efc7d2 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:39:37.573259 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcd3317-a2d7-4228-bdb0-e4ffc7efc7d2 None None] PATH_INFO: `/` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:39:37.574631 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 647/1321] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 10:39:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 10:39:37.580131 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9e696d-6aa4-4f42-953b-b5c10755f257 None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:39:37.580553 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9e696d-6aa4-4f42-953b-b5c10755f257 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:39:37.580699 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9e696d-6aa4-4f42-953b-b5c10755f257 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:39:37.653489 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-ea9e696d-6aa4-4f42-953b-b5c10755f257 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:39:37.654797 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-ea9e696d-6aa4-4f42-953b-b5c10755f257 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:39:37.717512 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:39:37.773035 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 675/1322] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 10:39:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:39:37.815272 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcd3317-a2d7-4228-bdb0-e4ffc7efc7d2 None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:39:37.816919 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-bfcd3317-a2d7-4228-bdb0-e4ffc7efc7d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:39:37.838748 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:39:37.929046 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcd3317-a2d7-4228-bdb0-e4ffc7efc7d2 None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:39:38.005154 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81566897-6681-401b-a167-f695254ba1af service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7922d0a1a7c4e808d0b2a80dca0da40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:39:38.005779 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81566897-6681-401b-a167-f695254ba1af service neutron] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:39:38.006058 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81566897-6681-401b-a167-f695254ba1af service neutron] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:39:38.006330 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81566897-6681-401b-a167-f695254ba1af service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:39:38.008316 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-81566897-6681-401b-a167-f695254ba1af service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:39:38.081440 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:39:38.123165 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81566897-6681-401b-a167-f695254ba1af service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:39:38.125825 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81566897-6681-401b-a167-f695254ba1af service neutron] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:39:38.147198 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 648/1323] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:39:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 10:39:40.229740 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19f0238-0a00-4bb8-bc86-f964a73bb92c None None] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:39:40.230250 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19f0238-0a00-4bb8-bc86-f964a73bb92c None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:39:40.230250 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19f0238-0a00-4bb8-bc86-f964a73bb92c None None] PATH_INFO: `/` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:39:40.231632 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 676/1324] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 10:39:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 10:39:40.237359 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fea24e-f393-4dff-ac9a-1f0a4ce4b16f None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:39:40.237779 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fea24e-f393-4dff-ac9a-1f0a4ce4b16f None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:39:40.238001 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fea24e-f393-4dff-ac9a-1f0a4ce4b16f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:39:40.305425 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-f9fea24e-f393-4dff-ac9a-1f0a4ce4b16f None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:39:40.306662 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-f9fea24e-f393-4dff-ac9a-1f0a4ce4b16f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:39:40.405407 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 649/1325] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 10:39:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 10:39:40.441935 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19f0238-0a00-4bb8-bc86-f964a73bb92c None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:39:40.443495 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-b19f0238-0a00-4bb8-bc86-f964a73bb92c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:39:40.546557 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19f0238-0a00-4bb8-bc86-f964a73bb92c None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:39:40.621032 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80f688a-4b3a-4711-bda4-31dfa712a76e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7922d0a1a7c4e808d0b2a80dca0da40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:39:40.621660 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80f688a-4b3a-4711-bda4-31dfa712a76e service neutron] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:39:40.621944 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80f688a-4b3a-4711-bda4-31dfa712a76e service neutron] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:39:40.622221 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80f688a-4b3a-4711-bda4-31dfa712a76e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:39:40.623919 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-c80f688a-4b3a-4711-bda4-31dfa712a76e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:39:40.643167 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:39:40.739451 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80f688a-4b3a-4711-bda4-31dfa712a76e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:39:40.740883 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80f688a-4b3a-4711-bda4-31dfa712a76e service neutron] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:39:40.759697 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 677/1326] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:39:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 10:39:42.851135 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5271ff12-4653-4c2d-b9ab-341a0e72548f None None] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:39:42.851721 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5271ff12-4653-4c2d-b9ab-341a0e72548f None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:39:42.851721 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5271ff12-4653-4c2d-b9ab-341a0e72548f None None] PATH_INFO: `/` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:39:42.853029 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 650/1327] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 10:39:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 10:39:42.858186 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa2d9f6-4c67-4173-89fa-772fc339508a None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:39:42.858573 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa2d9f6-4c67-4173-89fa-772fc339508a None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:39:42.858802 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa2d9f6-4c67-4173-89fa-772fc339508a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:39:42.933735 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-9aa2d9f6-4c67-4173-89fa-772fc339508a None None] MFA Rules not processed for user `6832ba6fe7d34e438a50d7f4d5681d00`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:39:42.934924 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-9aa2d9f6-4c67-4173-89fa-772fc339508a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:39:43.001633 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:39:43.036684 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 678/1328] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 10:39:42 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:39:43.074940 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5271ff12-4653-4c2d-b9ab-341a0e72548f None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:39:43.076562 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-5271ff12-4653-4c2d-b9ab-341a0e72548f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:39:43.107047 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:39:43.190436 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5271ff12-4653-4c2d-b9ab-341a0e72548f None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:39:43.268779 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b23f320-e342-4e01-80b3-c5d2cd57b5d7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7922d0a1a7c4e808d0b2a80dca0da40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:39:43.269410 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b23f320-e342-4e01-80b3-c5d2cd57b5d7 service neutron] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:39:43.269697 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b23f320-e342-4e01-80b3-c5d2cd57b5d7 service neutron] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:39:43.269971 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b23f320-e342-4e01-80b3-c5d2cd57b5d7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:39:43.271700 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-9b23f320-e342-4e01-80b3-c5d2cd57b5d7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:39:43.348640 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:39:43.381203 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b23f320-e342-4e01-80b3-c5d2cd57b5d7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:39:43.383008 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b23f320-e342-4e01-80b3-c5d2cd57b5d7 service neutron] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:39:43.401195 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 651/1329] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:39:43 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 10:39:45.692805 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26aa5a1b-49ba-479d-afc3-591813682dca None None] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:39:45.693332 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26aa5a1b-49ba-479d-afc3-591813682dca None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:39:45.693448 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26aa5a1b-49ba-479d-afc3-591813682dca None None] PATH_INFO: `/` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:39:45.694867 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 679/1330] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 10:39:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 10:39:45.700100 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86386ac1-7348-4099-b1bb-c337be522f3c None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:39:45.700489 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86386ac1-7348-4099-b1bb-c337be522f3c None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:39:45.700661 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86386ac1-7348-4099-b1bb-c337be522f3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:39:45.765958 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-86386ac1-7348-4099-b1bb-c337be522f3c None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:39:45.767179 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-86386ac1-7348-4099-b1bb-c337be522f3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:39:45.861395 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 652/1331] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 10:39:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:39:45.909809 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb2fe73-c13d-4be8-af68-ebb44f00b637 None None] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:39:45.910103 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb2fe73-c13d-4be8-af68-ebb44f00b637 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:39:45.910384 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb2fe73-c13d-4be8-af68-ebb44f00b637 None None] PATH_INFO: `/` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:39:45.911890 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 680/1332] 10.222.0.36 () {54 vars in 965 bytes} [Wed Jul 27 10:39:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 10:39:45.917604 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ed0cbd-fd3f-4656-83c0-0dfb07b42f78 None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:39:45.917894 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ed0cbd-fd3f-4656-83c0-0dfb07b42f78 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:39:45.918172 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ed0cbd-fd3f-4656-83c0-0dfb07b42f78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:39:45.965828 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:39:45.998164 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-83ed0cbd-fd3f-4656-83c0-0dfb07b42f78 None None] MFA Rules not processed for user `c7922d0a1a7c4e808d0b2a80dca0da40`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:39:45.999420 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-83ed0cbd-fd3f-4656-83c0-0dfb07b42f78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:39:46.075193 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 653/1333] 10.222.0.36 () {58 vars in 1062 bytes} [Wed Jul 27 10:39:45 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:39:46.084741 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb2fe73-c13d-4be8-af68-ebb44f00b637 None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:39:46.086356 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-8bb2fe73-c13d-4be8-af68-ebb44f00b637 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:39:46.188455 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb2fe73-c13d-4be8-af68-ebb44f00b637 None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:39:46.260139 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8057ab29-bfc2-4e90-80aa-d6d1a7e76640 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7922d0a1a7c4e808d0b2a80dca0da40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:39:46.260694 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8057ab29-bfc2-4e90-80aa-d6d1a7e76640 service neutron] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:39:46.260978 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8057ab29-bfc2-4e90-80aa-d6d1a7e76640 service neutron] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:39:46.261255 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8057ab29-bfc2-4e90-80aa-d6d1a7e76640 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:39:46.262903 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-8057ab29-bfc2-4e90-80aa-d6d1a7e76640 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:39:46.290676 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:39:46.372496 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8057ab29-bfc2-4e90-80aa-d6d1a7e76640 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:39:46.373925 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8057ab29-bfc2-4e90-80aa-d6d1a7e76640 service neutron] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:39:46.392679 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 681/1334] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:39:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 10:39:48.635305 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82044524-78a1-47f1-a426-794844552990 None None] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:39:48.635946 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82044524-78a1-47f1-a426-794844552990 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:39:48.635946 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82044524-78a1-47f1-a426-794844552990 None None] PATH_INFO: `/` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:39:48.637219 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 654/1335] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 10:39:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 10:39:48.642783 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09e03cf-0a3e-4bad-97e1-6745f310bee2 None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:39:48.643234 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09e03cf-0a3e-4bad-97e1-6745f310bee2 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:39:48.643338 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09e03cf-0a3e-4bad-97e1-6745f310bee2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:39:48.708600 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-b09e03cf-0a3e-4bad-97e1-6745f310bee2 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:39:48.709791 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-b09e03cf-0a3e-4bad-97e1-6745f310bee2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:39:48.783040 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:39:48.806036 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 682/1336] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 10:39:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:39:48.845278 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82044524-78a1-47f1-a426-794844552990 None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:39:48.846874 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-82044524-78a1-47f1-a426-794844552990 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:39:48.949662 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82044524-78a1-47f1-a426-794844552990 None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:39:48.977472 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:39:49.021820 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28441c4b-ae9d-4a6a-80f0-2d4cf95aa459 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7922d0a1a7c4e808d0b2a80dca0da40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:39:49.022434 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28441c4b-ae9d-4a6a-80f0-2d4cf95aa459 service neutron] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:39:49.022794 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28441c4b-ae9d-4a6a-80f0-2d4cf95aa459 service neutron] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:39:49.023069 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28441c4b-ae9d-4a6a-80f0-2d4cf95aa459 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:39:49.024691 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-28441c4b-ae9d-4a6a-80f0-2d4cf95aa459 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:39:49.130824 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28441c4b-ae9d-4a6a-80f0-2d4cf95aa459 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:39:49.132500 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28441c4b-ae9d-4a6a-80f0-2d4cf95aa459 service neutron] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:39:49.150542 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 655/1337] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:39:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 10:39:51.811170 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cbd74c-59f7-41e6-b4f4-cc04d8a3634d None None] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:39:51.811767 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cbd74c-59f7-41e6-b4f4-cc04d8a3634d None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:39:51.811767 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cbd74c-59f7-41e6-b4f4-cc04d8a3634d None None] PATH_INFO: `/` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:39:51.813085 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 683/1338] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 10:39:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 10:39:51.818157 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ba5b81-bf05-465e-99b2-0f045acad5f1 None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:39:51.818583 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ba5b81-bf05-465e-99b2-0f045acad5f1 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:39:51.818728 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ba5b81-bf05-465e-99b2-0f045acad5f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:39:51.883232 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-68ba5b81-bf05-465e-99b2-0f045acad5f1 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:39:51.884420 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-68ba5b81-bf05-465e-99b2-0f045acad5f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:39:51.892552 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:39:51.979003 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 656/1339] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 10:39:51 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 10:39:52.015475 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cbd74c-59f7-41e6-b4f4-cc04d8a3634d None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:39:52.017031 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-67cbd74c-59f7-41e6-b4f4-cc04d8a3634d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:39:52.124867 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cbd74c-59f7-41e6-b4f4-cc04d8a3634d None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:39:52.197470 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bf8722-aa9d-4c79-aba9-b2d22640abed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7922d0a1a7c4e808d0b2a80dca0da40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:39:52.198088 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bf8722-aa9d-4c79-aba9-b2d22640abed service neutron] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:39:52.198374 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bf8722-aa9d-4c79-aba9-b2d22640abed service neutron] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:39:52.198674 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bf8722-aa9d-4c79-aba9-b2d22640abed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:39:52.200328 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-38bf8722-aa9d-4c79-aba9-b2d22640abed service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:39:52.236151 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:39:52.309289 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38bf8722-aa9d-4c79-aba9-b2d22640abed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:39:52.310764 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38bf8722-aa9d-4c79-aba9-b2d22640abed service neutron] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:39:52.329163 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 684/1340] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:39:52 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 10:39:56.239230 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5095f5e-d2d4-4c75-a9be-ff8b35c5759f None None] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:39:56.239764 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5095f5e-d2d4-4c75-a9be-ff8b35c5759f None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:39:56.239764 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5095f5e-d2d4-4c75-a9be-ff8b35c5759f None None] PATH_INFO: `/` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:39:56.242624 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 657/1341] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 10:39:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 10:39:56.246327 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718c8911-4e27-4037-b39d-122248b19240 None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:39:56.246784 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718c8911-4e27-4037-b39d-122248b19240 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:39:56.246970 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718c8911-4e27-4037-b39d-122248b19240 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:39:56.314704 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-718c8911-4e27-4037-b39d-122248b19240 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:39:56.315933 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-718c8911-4e27-4037-b39d-122248b19240 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:39:56.402792 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:39:56.417260 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 685/1342] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 10:39:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:39:56.457465 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5095f5e-d2d4-4c75-a9be-ff8b35c5759f None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:39:56.459130 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-e5095f5e-d2d4-4c75-a9be-ff8b35c5759f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:39:56.567857 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5095f5e-d2d4-4c75-a9be-ff8b35c5759f None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:39:56.605064 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:39:56.646725 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62d2fd3-c854-4356-8cd7-48cd405977ce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7922d0a1a7c4e808d0b2a80dca0da40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:39:56.647347 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62d2fd3-c854-4356-8cd7-48cd405977ce service neutron] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:39:56.647626 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62d2fd3-c854-4356-8cd7-48cd405977ce service neutron] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:39:56.647898 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62d2fd3-c854-4356-8cd7-48cd405977ce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:39:56.649517 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-c62d2fd3-c854-4356-8cd7-48cd405977ce service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:39:56.762495 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62d2fd3-c854-4356-8cd7-48cd405977ce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:39:56.764334 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62d2fd3-c854-4356-8cd7-48cd405977ce service neutron] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:39:56.793879 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 658/1343] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:39:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 10:39:59.483227 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad5a52b-aaeb-4436-b369-9e8d2cbc3400 None None] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:39:59.483751 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad5a52b-aaeb-4436-b369-9e8d2cbc3400 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:39:59.483751 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad5a52b-aaeb-4436-b369-9e8d2cbc3400 None None] PATH_INFO: `/` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:39:59.485142 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 686/1344] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 10:39:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 10:39:59.490860 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f65641b-547d-46a6-b711-eb79f0111f34 None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:39:59.491367 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f65641b-547d-46a6-b711-eb79f0111f34 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:39:59.491464 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f65641b-547d-46a6-b711-eb79f0111f34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:39:59.564595 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-5f65641b-547d-46a6-b711-eb79f0111f34 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:39:59.565794 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-5f65641b-547d-46a6-b711-eb79f0111f34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:39:59.585678 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:39:59.666911 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 659/1345] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 10:39:59 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:39:59.705151 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad5a52b-aaeb-4436-b369-9e8d2cbc3400 None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:39:59.706811 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-5ad5a52b-aaeb-4436-b369-9e8d2cbc3400 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:39:59.817677 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad5a52b-aaeb-4436-b369-9e8d2cbc3400 None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:39:59.892324 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8673c6a7-8417-4680-a8fc-f2ab75d1cb07 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7922d0a1a7c4e808d0b2a80dca0da40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:39:59.892941 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8673c6a7-8417-4680-a8fc-f2ab75d1cb07 service neutron] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:39:59.893219 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8673c6a7-8417-4680-a8fc-f2ab75d1cb07 service neutron] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:39:59.893490 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8673c6a7-8417-4680-a8fc-f2ab75d1cb07 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:39:59.895127 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-8673c6a7-8417-4680-a8fc-f2ab75d1cb07 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:39:59.937263 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:40:00.007025 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8673c6a7-8417-4680-a8fc-f2ab75d1cb07 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:40:00.008460 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8673c6a7-8417-4680-a8fc-f2ab75d1cb07 service neutron] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:40:00.026737 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 687/1346] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:39:59 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 10:40:04.393557 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e77f995-637c-413a-bc35-58e61b8b0f9c None None] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:40:04.394072 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e77f995-637c-413a-bc35-58e61b8b0f9c None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:40:04.394072 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e77f995-637c-413a-bc35-58e61b8b0f9c None None] PATH_INFO: `/` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:40:04.395429 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 660/1347] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 10:40:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 10:40:04.400806 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efdc021-7a62-44a6-9b13-3e6b66207468 None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:40:04.401270 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efdc021-7a62-44a6-9b13-3e6b66207468 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:40:04.401391 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efdc021-7a62-44a6-9b13-3e6b66207468 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:40:04.471453 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-8efdc021-7a62-44a6-9b13-3e6b66207468 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:40:04.472679 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-8efdc021-7a62-44a6-9b13-3e6b66207468 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:40:04.570535 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:40:04.573614 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 688/1348] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 10:40:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:40:04.612223 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e77f995-637c-413a-bc35-58e61b8b0f9c None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:40:04.613826 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-6e77f995-637c-413a-bc35-58e61b8b0f9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:40:04.722823 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e77f995-637c-413a-bc35-58e61b8b0f9c None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:40:04.771255 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:40:04.801562 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3d9b10-b61f-4c64-8410-1547b5e8b1b8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7922d0a1a7c4e808d0b2a80dca0da40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:40:04.802189 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3d9b10-b61f-4c64-8410-1547b5e8b1b8 service neutron] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:40:04.802470 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3d9b10-b61f-4c64-8410-1547b5e8b1b8 service neutron] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:40:04.802784 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3d9b10-b61f-4c64-8410-1547b5e8b1b8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:40:04.804395 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-ce3d9b10-b61f-4c64-8410-1547b5e8b1b8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:40:04.914892 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3d9b10-b61f-4c64-8410-1547b5e8b1b8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:40:04.916257 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3d9b10-b61f-4c64-8410-1547b5e8b1b8 service neutron] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:40:04.934421 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 661/1349] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:40:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 10:40:09.847674 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c976bb-67ae-48bd-8f90-4dca952e2ad1 None None] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:40:09.847674 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c976bb-67ae-48bd-8f90-4dca952e2ad1 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:40:09.848212 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c976bb-67ae-48bd-8f90-4dca952e2ad1 None None] PATH_INFO: `/` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:40:09.849773 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 689/1350] 10.222.0.36 () {54 vars in 921 bytes} [Wed Jul 27 10:40:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 10:40:09.854903 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b3bc1c-fb92-445e-ad62-8388f8e05222 None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:40:09.855338 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b3bc1c-fb92-445e-ad62-8388f8e05222 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:40:09.855436 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b3bc1c-fb92-445e-ad62-8388f8e05222 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:40:09.985114 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-f1b3bc1c-fb92-445e-ad62-8388f8e05222 None None] MFA Rules not processed for user `16cc6179e9a04b6398fbacb40ad1c555`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:40:09.986341 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-f1b3bc1c-fb92-445e-ad62-8388f8e05222 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:40:10.023191 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:40:10.134104 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 662/1351] 10.222.0.36 () {58 vars in 1018 bytes} [Wed Jul 27 10:40:09 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:40:14.282225 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207e6ffe-9c80-42ac-b293-3c052ba2a0c2 None None] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:40:14.282762 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207e6ffe-9c80-42ac-b293-3c052ba2a0c2 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:40:14.282762 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207e6ffe-9c80-42ac-b293-3c052ba2a0c2 None None] PATH_INFO: `/` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:40:14.284067 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 690/1352] 10.222.0.36 () {54 vars in 921 bytes} [Wed Jul 27 10:40:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 10:40:14.289270 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56440555-9ef7-4f5a-a50b-acf43ddfa35c None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:40:14.289702 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56440555-9ef7-4f5a-a50b-acf43ddfa35c None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:40:14.289802 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56440555-9ef7-4f5a-a50b-acf43ddfa35c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:40:14.369555 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-56440555-9ef7-4f5a-a50b-acf43ddfa35c None None] MFA Rules not processed for user `16cc6179e9a04b6398fbacb40ad1c555`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:40:14.370770 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-56440555-9ef7-4f5a-a50b-acf43ddfa35c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:40:14.450211 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:40:14.493616 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 663/1353] 10.222.0.36 () {58 vars in 1018 bytes} [Wed Jul 27 10:40:14 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:40:15.810190 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbc8520-9546-4e9f-acac-8967e6f5be12 None None] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:40:15.810762 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbc8520-9546-4e9f-acac-8967e6f5be12 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:40:15.810762 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbc8520-9546-4e9f-acac-8967e6f5be12 None None] PATH_INFO: `/` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:40:15.812179 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 691/1354] 10.222.0.36 () {54 vars in 921 bytes} [Wed Jul 27 10:40:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 10:40:15.817111 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b74b2d-d6f9-473d-a257-b35748814826 None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:40:15.817558 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b74b2d-d6f9-473d-a257-b35748814826 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:40:15.817917 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b74b2d-d6f9-473d-a257-b35748814826 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:40:15.899478 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-62b74b2d-d6f9-473d-a257-b35748814826 None None] MFA Rules not processed for user `16cc6179e9a04b6398fbacb40ad1c555`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:40:15.900683 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-62b74b2d-d6f9-473d-a257-b35748814826 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:40:16.009630 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 664/1355] 10.222.0.36 () {58 vars in 1018 bytes} [Wed Jul 27 10:40:15 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 10:40:19.410978 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ba1227-6a05-41ef-981f-9487f1a7372b None None] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:40:19.411553 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ba1227-6a05-41ef-981f-9487f1a7372b None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:40:19.411553 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ba1227-6a05-41ef-981f-9487f1a7372b None None] PATH_INFO: `/` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:40:19.413951 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 692/1356] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 10:40:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 10:40:19.420182 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fd9ead-22e6-43e7-b404-c587cf3c040c None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:40:19.420662 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fd9ead-22e6-43e7-b404-c587cf3c040c None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:40:19.420878 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fd9ead-22e6-43e7-b404-c587cf3c040c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:40:19.460457 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:40:19.500198 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-77fd9ead-22e6-43e7-b404-c587cf3c040c None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:40:19.502565 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-77fd9ead-22e6-43e7-b404-c587cf3c040c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:40:19.609387 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 665/1357] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 10:40:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:40:19.665332 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be31f509-e1a9-46d2-b2bf-d4948c1fea54 None None] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:40:19.665625 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be31f509-e1a9-46d2-b2bf-d4948c1fea54 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:40:19.665903 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be31f509-e1a9-46d2-b2bf-d4948c1fea54 None None] PATH_INFO: `/` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:40:19.667288 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 693/1358] 10.222.0.36 () {54 vars in 962 bytes} [Wed Jul 27 10:40:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 10:40:19.681150 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b977e8-3cbe-4293-8dae-54cc186f0c53 None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:40:19.681520 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b977e8-3cbe-4293-8dae-54cc186f0c53 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:40:19.681859 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b977e8-3cbe-4293-8dae-54cc186f0c53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:40:19.768289 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-40b977e8-3cbe-4293-8dae-54cc186f0c53 None None] MFA Rules not processed for user `e059d8b6c28c4fcba925d05d7c98e4ac`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:40:19.769383 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-40b977e8-3cbe-4293-8dae-54cc186f0c53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:40:19.821747 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:40:19.894328 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 666/1359] 10.222.0.36 () {58 vars in 1059 bytes} [Wed Jul 27 10:40:19 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:40:19.902023 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be31f509-e1a9-46d2-b2bf-d4948c1fea54 None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:40:19.903660 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-be31f509-e1a9-46d2-b2bf-d4948c1fea54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:40:20.089218 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be31f509-e1a9-46d2-b2bf-d4948c1fea54 None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:40:20.170677 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:40:20.217935 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9357f15f-40aa-4293-a263-c9057092acae service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e059d8b6c28c4fcba925d05d7c98e4ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:40:20.218564 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9357f15f-40aa-4293-a263-c9057092acae service nova] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:40:20.218955 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9357f15f-40aa-4293-a263-c9057092acae service nova] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:40:20.219241 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9357f15f-40aa-4293-a263-c9057092acae service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:40:20.221017 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-9357f15f-40aa-4293-a263-c9057092acae service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:40:20.321584 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9357f15f-40aa-4293-a263-c9057092acae service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:40:20.323412 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9357f15f-40aa-4293-a263-c9057092acae service nova] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:40:20.345009 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 694/1360] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:40:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 443 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 10:40:22.559373 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc24d406-2b71-4108-a5c4-6cd3aa8a5b70 None None] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:40:22.559792 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc24d406-2b71-4108-a5c4-6cd3aa8a5b70 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:40:22.560003 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc24d406-2b71-4108-a5c4-6cd3aa8a5b70 None None] PATH_INFO: `/` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:40:22.561430 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 667/1361] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 10:40:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 10:40:22.566862 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da829291-1af8-4055-a8c0-e5ae45e9ff97 None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:40:22.567298 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da829291-1af8-4055-a8c0-e5ae45e9ff97 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:40:22.567405 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da829291-1af8-4055-a8c0-e5ae45e9ff97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:40:22.636565 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-da829291-1af8-4055-a8c0-e5ae45e9ff97 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:40:22.637771 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-da829291-1af8-4055-a8c0-e5ae45e9ff97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:40:22.676661 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:40:22.746578 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 695/1362] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 10:40:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:40:22.792024 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc24d406-2b71-4108-a5c4-6cd3aa8a5b70 None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:40:22.793663 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-bc24d406-2b71-4108-a5c4-6cd3aa8a5b70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:40:22.968625 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc24d406-2b71-4108-a5c4-6cd3aa8a5b70 None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:40:22.983358 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:40:23.133106 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6e1f2d-7898-47d9-a37a-7172449b0951 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e059d8b6c28c4fcba925d05d7c98e4ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:40:23.133735 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6e1f2d-7898-47d9-a37a-7172449b0951 service nova] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:40:23.134019 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6e1f2d-7898-47d9-a37a-7172449b0951 service nova] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:40:23.134295 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6e1f2d-7898-47d9-a37a-7172449b0951 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:40:23.135976 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-2d6e1f2d-7898-47d9-a37a-7172449b0951 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:40:23.229181 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6e1f2d-7898-47d9-a37a-7172449b0951 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:40:23.230822 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6e1f2d-7898-47d9-a37a-7172449b0951 service nova] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:40:23.250318 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 668/1363] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:40:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 459 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 10:40:25.658756 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aed03aa-12d2-4457-9108-a8d8fbff6e7c None None] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:40:25.659299 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aed03aa-12d2-4457-9108-a8d8fbff6e7c None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:40:25.659299 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aed03aa-12d2-4457-9108-a8d8fbff6e7c None None] PATH_INFO: `/` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:40:25.661219 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 696/1364] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 10:40:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 10:40:25.666579 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3680fd7d-337d-4644-8fd8-234ccfac372e None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:40:25.667094 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3680fd7d-337d-4644-8fd8-234ccfac372e None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:40:25.667232 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3680fd7d-337d-4644-8fd8-234ccfac372e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:40:25.686479 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:40:25.744002 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-3680fd7d-337d-4644-8fd8-234ccfac372e None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:40:25.745226 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-3680fd7d-337d-4644-8fd8-234ccfac372e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:40:25.847913 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 669/1365] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 10:40:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:40:25.893447 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f429563-d8fb-4156-ba55-0184fe3ef6e1 None None] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:40:25.893737 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f429563-d8fb-4156-ba55-0184fe3ef6e1 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:40:25.894038 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f429563-d8fb-4156-ba55-0184fe3ef6e1 None None] PATH_INFO: `/` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:40:25.895507 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 697/1366] 10.222.0.36 () {54 vars in 962 bytes} [Wed Jul 27 10:40:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 10:40:25.900783 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaf67ff-a514-4a2e-b019-3d5c287da959 None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:40:25.901068 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaf67ff-a514-4a2e-b019-3d5c287da959 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:40:25.901342 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaf67ff-a514-4a2e-b019-3d5c287da959 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:40:25.989697 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-2eaf67ff-a514-4a2e-b019-3d5c287da959 None None] MFA Rules not processed for user `e059d8b6c28c4fcba925d05d7c98e4ac`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:40:25.990907 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-2eaf67ff-a514-4a2e-b019-3d5c287da959 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:40:26.027861 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:40:26.116405 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 670/1367] 10.222.0.36 () {58 vars in 1059 bytes} [Wed Jul 27 10:40:25 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:40:26.124393 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f429563-d8fb-4156-ba55-0184fe3ef6e1 None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:40:26.125902 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-8f429563-d8fb-4156-ba55-0184fe3ef6e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:40:26.285021 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f429563-d8fb-4156-ba55-0184fe3ef6e1 None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:40:26.306236 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:40:26.404344 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c603105-97ba-477f-b8f7-3661ff6a96ae service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e059d8b6c28c4fcba925d05d7c98e4ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:40:26.405605 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c603105-97ba-477f-b8f7-3661ff6a96ae service nova] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:40:26.405892 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c603105-97ba-477f-b8f7-3661ff6a96ae service nova] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:40:26.406162 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c603105-97ba-477f-b8f7-3661ff6a96ae service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:40:26.407811 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-5c603105-97ba-477f-b8f7-3661ff6a96ae service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:40:26.500905 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c603105-97ba-477f-b8f7-3661ff6a96ae service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:40:26.502571 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c603105-97ba-477f-b8f7-3661ff6a96ae service nova] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:40:26.521918 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 698/1368] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:40:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 10:40:27.532703 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3580828-7f58-4ef7-a7ee-dfe39b260b62 None None] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:40:27.533257 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3580828-7f58-4ef7-a7ee-dfe39b260b62 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:40:27.533257 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3580828-7f58-4ef7-a7ee-dfe39b260b62 None None] PATH_INFO: `/` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:40:27.534811 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 671/1369] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 10:40:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 10:40:27.539788 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7a8a51-035a-450b-85c7-ab649098daf7 None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:40:27.540229 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7a8a51-035a-450b-85c7-ab649098daf7 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:40:27.540329 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7a8a51-035a-450b-85c7-ab649098daf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:40:27.573485 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:40:27.611822 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-2d7a8a51-035a-450b-85c7-ab649098daf7 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:40:27.613095 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-2d7a8a51-035a-450b-85c7-ab649098daf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:40:27.731813 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 699/1370] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 10:40:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:40:28.042307 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdd4db6-e261-4295-a212-058481cc6e14 None None] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:40:28.042683 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdd4db6-e261-4295-a212-058481cc6e14 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:40:28.042912 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdd4db6-e261-4295-a212-058481cc6e14 None None] PATH_INFO: `/` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:40:28.044519 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 672/1371] 10.222.0.36 () {54 vars in 964 bytes} [Wed Jul 27 10:40:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 10:40:28.050039 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5bc60b-e2a6-4091-9564-840793747cb6 None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:40:28.050338 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5bc60b-e2a6-4091-9564-840793747cb6 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:40:28.050632 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5bc60b-e2a6-4091-9564-840793747cb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:40:28.158059 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-1e5bc60b-e2a6-4091-9564-840793747cb6 None None] MFA Rules not processed for user `864d35f268cf45148756109759bb2743`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:40:28.159306 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-1e5bc60b-e2a6-4091-9564-840793747cb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:40:28.213250 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:40:28.247670 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 700/1372] 10.222.0.36 () {58 vars in 1061 bytes} [Wed Jul 27 10:40:28 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:40:28.255277 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdd4db6-e261-4295-a212-058481cc6e14 None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:40:28.257050 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-8cdd4db6-e261-4295-a212-058481cc6e14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:40:28.373062 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdd4db6-e261-4295-a212-058481cc6e14 None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:40:28.417388 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:40:28.456352 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ea6997-3902-46b3-83aa-762074d64a3b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '864d35f268cf45148756109759bb2743', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:40:28.457114 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ea6997-3902-46b3-83aa-762074d64a3b service cinder] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:40:28.457403 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ea6997-3902-46b3-83aa-762074d64a3b service cinder] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:40:28.457681 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ea6997-3902-46b3-83aa-762074d64a3b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:40:28.459377 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-12ea6997-3902-46b3-83aa-762074d64a3b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:40:28.586851 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3a9bbb-8741-4775-ae8c-5373a63831af None None] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:40:28.587288 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3a9bbb-8741-4775-ae8c-5373a63831af None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:40:28.587288 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3a9bbb-8741-4775-ae8c-5373a63831af None None] PATH_INFO: `/` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:40:28.588644 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 701/1373] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 10:40:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 10:40:28.594223 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ea331c-7fa2-4998-9281-78fc6c99da4c None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:40:28.594508 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ea331c-7fa2-4998-9281-78fc6c99da4c None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:40:28.594834 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ea331c-7fa2-4998-9281-78fc6c99da4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:40:28.599981 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ea6997-3902-46b3-83aa-762074d64a3b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:40:28.602488 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ea6997-3902-46b3-83aa-762074d64a3b service cinder] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:40:28.630770 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 673/1374] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:40:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 10:40:28.670149 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-a2ea331c-7fa2-4998-9281-78fc6c99da4c None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:40:28.671457 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-a2ea331c-7fa2-4998-9281-78fc6c99da4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:40:28.791763 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 702/1375] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 10:40:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:40:28.840956 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ea6997-3902-46b3-83aa-762074d64a3b service cinder] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:40:28.842717 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-12ea6997-3902-46b3-83aa-762074d64a3b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:40:28.925920 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:40:28.973394 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ea6997-3902-46b3-83aa-762074d64a3b service cinder] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:40:29.094417 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88c5173-6d51-4654-adb7-e34e056cb960 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e059d8b6c28c4fcba925d05d7c98e4ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:40:29.095064 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88c5173-6d51-4654-adb7-e34e056cb960 service nova] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:40:29.095339 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88c5173-6d51-4654-adb7-e34e056cb960 service nova] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:40:29.095606 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88c5173-6d51-4654-adb7-e34e056cb960 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:40:29.097211 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-b88c5173-6d51-4654-adb7-e34e056cb960 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:40:29.177247 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:40:29.188166 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88c5173-6d51-4654-adb7-e34e056cb960 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:40:29.189769 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88c5173-6d51-4654-adb7-e34e056cb960 service nova] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:40:29.207855 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 674/1376] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:40:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 10:40:31.243432 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06508ac9-80aa-44b5-8f31-da4c1480b7e6 None None] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:40:31.243889 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06508ac9-80aa-44b5-8f31-da4c1480b7e6 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:40:31.243982 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06508ac9-80aa-44b5-8f31-da4c1480b7e6 None None] PATH_INFO: `/` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:40:31.245423 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 703/1377] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 10:40:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 10:40:31.250718 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fc8b58-6e64-4df1-b920-503d5f8dc3b1 None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:40:31.251165 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fc8b58-6e64-4df1-b920-503d5f8dc3b1 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:40:31.251304 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fc8b58-6e64-4df1-b920-503d5f8dc3b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:40:31.327957 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-58fc8b58-6e64-4df1-b920-503d5f8dc3b1 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:40:31.329188 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-58fc8b58-6e64-4df1-b920-503d5f8dc3b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:40:31.435377 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 675/1378] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 10:40:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:40:31.479864 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06508ac9-80aa-44b5-8f31-da4c1480b7e6 None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:40:31.481474 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-06508ac9-80aa-44b5-8f31-da4c1480b7e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:40:31.537020 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:40:31.636619 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06508ac9-80aa-44b5-8f31-da4c1480b7e6 None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:40:31.771531 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63dd2e8-9f5e-4f5b-9cfa-1f458b6dfc71 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e059d8b6c28c4fcba925d05d7c98e4ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:40:31.772087 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63dd2e8-9f5e-4f5b-9cfa-1f458b6dfc71 service nova] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:40:31.772369 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63dd2e8-9f5e-4f5b-9cfa-1f458b6dfc71 service nova] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:40:31.772645 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63dd2e8-9f5e-4f5b-9cfa-1f458b6dfc71 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:40:31.774299 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-c63dd2e8-9f5e-4f5b-9cfa-1f458b6dfc71 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:40:31.788333 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:40:31.877516 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63dd2e8-9f5e-4f5b-9cfa-1f458b6dfc71 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:40:31.879278 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63dd2e8-9f5e-4f5b-9cfa-1f458b6dfc71 service nova] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:40:31.898845 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 704/1379] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:40:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 10:40:33.955182 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2dcdba-f2cc-4b0b-8ba7-384718fcfd99 None None] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:40:33.955642 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2dcdba-f2cc-4b0b-8ba7-384718fcfd99 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:40:33.955830 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2dcdba-f2cc-4b0b-8ba7-384718fcfd99 None None] PATH_INFO: `/` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:40:33.957423 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 676/1380] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 10:40:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 10:40:33.962594 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc15561c-3a66-4f90-aa51-397aa7f87f07 None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:40:33.963020 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc15561c-3a66-4f90-aa51-397aa7f87f07 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:40:33.963200 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc15561c-3a66-4f90-aa51-397aa7f87f07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:40:34.032470 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-bc15561c-3a66-4f90-aa51-397aa7f87f07 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:40:34.033961 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-bc15561c-3a66-4f90-aa51-397aa7f87f07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:40:34.135245 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:40:34.149004 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 705/1381] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 10:40:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:40:34.195963 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2dcdba-f2cc-4b0b-8ba7-384718fcfd99 None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:40:34.197485 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-ad2dcdba-f2cc-4b0b-8ba7-384718fcfd99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:40:34.265842 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:40:34.371926 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2dcdba-f2cc-4b0b-8ba7-384718fcfd99 None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:40:34.497199 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc187d4-6872-44dc-bfc3-abf262835ad8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e059d8b6c28c4fcba925d05d7c98e4ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:40:34.497732 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc187d4-6872-44dc-bfc3-abf262835ad8 service nova] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:40:34.498018 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc187d4-6872-44dc-bfc3-abf262835ad8 service nova] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:40:34.498409 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc187d4-6872-44dc-bfc3-abf262835ad8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:40:34.500236 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-bfc187d4-6872-44dc-bfc3-abf262835ad8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:40:34.523940 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:40:34.598989 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc187d4-6872-44dc-bfc3-abf262835ad8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:40:34.600617 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc187d4-6872-44dc-bfc3-abf262835ad8 service nova] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:40:34.621947 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 677/1382] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:40:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 10:40:36.676214 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ee3ff4-20c8-444a-9ebf-d8d702f0ead5 None None] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:40:36.676747 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ee3ff4-20c8-444a-9ebf-d8d702f0ead5 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:40:36.676844 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ee3ff4-20c8-444a-9ebf-d8d702f0ead5 None None] PATH_INFO: `/` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:40:36.678151 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 706/1383] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 10:40:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 10:40:36.683667 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e45863a-0eaf-4e80-97c6-06d60d999465 None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:40:36.684104 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e45863a-0eaf-4e80-97c6-06d60d999465 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:40:36.684235 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e45863a-0eaf-4e80-97c6-06d60d999465 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:40:36.756764 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-2e45863a-0eaf-4e80-97c6-06d60d999465 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:40:36.757978 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-2e45863a-0eaf-4e80-97c6-06d60d999465 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:40:36.851632 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:40:36.856027 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 678/1384] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 10:40:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:40:36.905086 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ee3ff4-20c8-444a-9ebf-d8d702f0ead5 None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:40:36.907198 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-a8ee3ff4-20c8-444a-9ebf-d8d702f0ead5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:40:37.066015 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ee3ff4-20c8-444a-9ebf-d8d702f0ead5 None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:40:37.131189 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:40:37.188859 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39d567a-e16a-43b9-8b05-b89aa6f310a3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e059d8b6c28c4fcba925d05d7c98e4ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:40:37.189481 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39d567a-e16a-43b9-8b05-b89aa6f310a3 service nova] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:40:37.189762 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39d567a-e16a-43b9-8b05-b89aa6f310a3 service nova] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:40:37.190036 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39d567a-e16a-43b9-8b05-b89aa6f310a3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:40:37.191708 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-b39d567a-e16a-43b9-8b05-b89aa6f310a3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:40:37.288711 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39d567a-e16a-43b9-8b05-b89aa6f310a3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:40:37.290324 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39d567a-e16a-43b9-8b05-b89aa6f310a3 service nova] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:40:37.308677 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 707/1385] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:40:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 10:40:39.634614 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb41c52-e96c-4900-9dc0-bc49a801e8bb None None] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:40:39.635180 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb41c52-e96c-4900-9dc0-bc49a801e8bb None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:40:39.635369 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb41c52-e96c-4900-9dc0-bc49a801e8bb None None] PATH_INFO: `/` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:40:39.637088 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 679/1386] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 10:40:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 10:40:39.642567 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0235b70e-03c6-4d2c-89b2-d046b316303c None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:40:39.643000 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0235b70e-03c6-4d2c-89b2-d046b316303c None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:40:39.643202 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0235b70e-03c6-4d2c-89b2-d046b316303c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:40:39.709104 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-0235b70e-03c6-4d2c-89b2-d046b316303c None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:40:39.710323 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-0235b70e-03c6-4d2c-89b2-d046b316303c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:40:39.735092 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:40:39.811641 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 708/1387] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 10:40:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:40:39.856692 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb41c52-e96c-4900-9dc0-bc49a801e8bb None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:40:39.858980 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-edb41c52-e96c-4900-9dc0-bc49a801e8bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:40:40.005158 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb41c52-e96c-4900-9dc0-bc49a801e8bb None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:40:40.082797 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:40:40.126875 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802df431-45f0-4381-a4e2-b6b2977d77e1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e059d8b6c28c4fcba925d05d7c98e4ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:40:40.127511 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802df431-45f0-4381-a4e2-b6b2977d77e1 service nova] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:40:40.127789 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802df431-45f0-4381-a4e2-b6b2977d77e1 service nova] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:40:40.128062 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802df431-45f0-4381-a4e2-b6b2977d77e1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:40:40.129701 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-802df431-45f0-4381-a4e2-b6b2977d77e1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:40:40.226529 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802df431-45f0-4381-a4e2-b6b2977d77e1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:40:40.228171 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802df431-45f0-4381-a4e2-b6b2977d77e1 service nova] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:40:40.247221 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 680/1388] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:40:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 10:40:42.285269 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b69b96-0490-4266-a3eb-74f92d79c31d None None] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:40:42.285931 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b69b96-0490-4266-a3eb-74f92d79c31d None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:40:42.285931 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b69b96-0490-4266-a3eb-74f92d79c31d None None] PATH_INFO: `/` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:40:42.287245 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 709/1389] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 10:40:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 10:40:42.292513 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6539fe77-d890-46bc-92d3-df7fa6a8708e None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:40:42.292961 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6539fe77-d890-46bc-92d3-df7fa6a8708e None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:40:42.293075 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6539fe77-d890-46bc-92d3-df7fa6a8708e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:40:42.360235 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-6539fe77-d890-46bc-92d3-df7fa6a8708e None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:40:42.361507 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-6539fe77-d890-46bc-92d3-df7fa6a8708e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:40:42.398649 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:40:42.463337 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 681/1390] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 10:40:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:40:42.513646 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b69b96-0490-4266-a3eb-74f92d79c31d None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:40:42.515553 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-03b69b96-0490-4266-a3eb-74f92d79c31d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:40:42.674663 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b69b96-0490-4266-a3eb-74f92d79c31d None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:40:42.684028 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:40:42.792049 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fca5e10-9901-4ad6-a18e-627c9723fb09 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e059d8b6c28c4fcba925d05d7c98e4ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:40:42.792681 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fca5e10-9901-4ad6-a18e-627c9723fb09 service nova] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:40:42.792965 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fca5e10-9901-4ad6-a18e-627c9723fb09 service nova] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:40:42.793241 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fca5e10-9901-4ad6-a18e-627c9723fb09 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:40:42.794914 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-4fca5e10-9901-4ad6-a18e-627c9723fb09 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:40:42.887704 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fca5e10-9901-4ad6-a18e-627c9723fb09 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:40:42.889362 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fca5e10-9901-4ad6-a18e-627c9723fb09 service nova] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:40:42.908202 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 710/1391] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:40:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 10:40:45.317621 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c020730e-a0ed-449e-b16e-dcf73e012a7c None None] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:40:45.318150 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c020730e-a0ed-449e-b16e-dcf73e012a7c None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:40:45.318261 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c020730e-a0ed-449e-b16e-dcf73e012a7c None None] PATH_INFO: `/` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:40:45.319837 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 682/1392] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 10:40:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 10:40:45.325077 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8bb210-976c-4c4a-9fa5-d1caf023b918 None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:40:45.325507 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8bb210-976c-4c4a-9fa5-d1caf023b918 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:40:45.325762 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8bb210-976c-4c4a-9fa5-d1caf023b918 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:40:45.341206 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:40:45.393684 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-5e8bb210-976c-4c4a-9fa5-d1caf023b918 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:40:45.394939 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-5e8bb210-976c-4c4a-9fa5-d1caf023b918 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:40:45.492828 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 711/1393] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 10:40:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 10:40:45.536917 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c020730e-a0ed-449e-b16e-dcf73e012a7c None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:40:45.538430 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-c020730e-a0ed-449e-b16e-dcf73e012a7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:40:45.690234 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c020730e-a0ed-449e-b16e-dcf73e012a7c None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:40:45.707842 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:40:45.810032 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cfe075-3d45-4d02-87dc-468875585f91 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e059d8b6c28c4fcba925d05d7c98e4ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:40:45.810771 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cfe075-3d45-4d02-87dc-468875585f91 service nova] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:40:45.811056 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cfe075-3d45-4d02-87dc-468875585f91 service nova] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:40:45.811347 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cfe075-3d45-4d02-87dc-468875585f91 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:40:45.812960 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-e9cfe075-3d45-4d02-87dc-468875585f91 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:40:45.901582 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9cfe075-3d45-4d02-87dc-468875585f91 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:40:45.903217 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9cfe075-3d45-4d02-87dc-468875585f91 service nova] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:40:45.922954 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 683/1394] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:40:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 10:40:46.975613 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d670e3d0-953f-4eaa-9c58-3694ad7084e0 None None] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:40:46.976160 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d670e3d0-953f-4eaa-9c58-3694ad7084e0 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:40:46.976257 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d670e3d0-953f-4eaa-9c58-3694ad7084e0 None None] PATH_INFO: `/` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:40:46.977547 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 712/1395] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 10:40:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 10:40:46.983377 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53264bc0-70ef-4877-ae79-7a36b4d8a6e1 None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:40:46.983775 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53264bc0-70ef-4877-ae79-7a36b4d8a6e1 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:40:46.983931 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53264bc0-70ef-4877-ae79-7a36b4d8a6e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:40:47.010087 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:40:47.050308 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-53264bc0-70ef-4877-ae79-7a36b4d8a6e1 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:40:47.051520 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-53264bc0-70ef-4877-ae79-7a36b4d8a6e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:40:47.150097 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 684/1396] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 10:40:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:40:47.205885 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd35ea5a-56ae-424d-95a4-b186b1a86b3b None None] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:40:47.206167 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd35ea5a-56ae-424d-95a4-b186b1a86b3b None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:40:47.206448 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd35ea5a-56ae-424d-95a4-b186b1a86b3b None None] PATH_INFO: `/` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:40:47.207836 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 713/1397] 10.222.0.36 () {54 vars in 966 bytes} [Wed Jul 27 10:40:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 10:40:47.212983 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3ffecb-987d-48c7-97aa-36eaab036aa6 None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:40:47.213272 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3ffecb-987d-48c7-97aa-36eaab036aa6 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:40:47.213542 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3ffecb-987d-48c7-97aa-36eaab036aa6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:40:47.289712 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-1a3ffecb-987d-48c7-97aa-36eaab036aa6 None None] MFA Rules not processed for user `bc03c5d232654878aa1ddce15f49fdaa`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:40:47.290947 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-1a3ffecb-987d-48c7-97aa-36eaab036aa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:40:47.334911 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:40:47.371858 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 685/1398] 10.222.0.36 () {58 vars in 1063 bytes} [Wed Jul 27 10:40:47 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:40:47.381574 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd35ea5a-56ae-424d-95a4-b186b1a86b3b None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:40:47.383444 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-fd35ea5a-56ae-424d-95a4-b186b1a86b3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:40:47.499461 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:40:47.507443 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd35ea5a-56ae-424d-95a4-b186b1a86b3b None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:40:47.587906 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a8633d-e1b1-4eaf-a892-29c0eabaef9a service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc03c5d232654878aa1ddce15f49fdaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:40:47.588555 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a8633d-e1b1-4eaf-a892-29c0eabaef9a service glance] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:40:47.588837 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a8633d-e1b1-4eaf-a892-29c0eabaef9a service glance] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:40:47.589112 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a8633d-e1b1-4eaf-a892-29c0eabaef9a service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:40:47.590844 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-81a8633d-e1b1-4eaf-a892-29c0eabaef9a service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:40:47.705554 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a8633d-e1b1-4eaf-a892-29c0eabaef9a service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:40:47.707020 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a8633d-e1b1-4eaf-a892-29c0eabaef9a service glance] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:40:47.725959 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 714/1399] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:40:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 10:40:48.042978 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3970207a-27ab-43a5-a054-4f159918ced2 None None] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:40:48.043631 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3970207a-27ab-43a5-a054-4f159918ced2 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:40:48.043631 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3970207a-27ab-43a5-a054-4f159918ced2 None None] PATH_INFO: `/` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:40:48.044824 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 686/1400] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 10:40:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 10:40:48.050223 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2903a1-02c4-4dae-910e-a0bc3c168427 None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:40:48.050701 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2903a1-02c4-4dae-910e-a0bc3c168427 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:40:48.050807 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2903a1-02c4-4dae-910e-a0bc3c168427 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:40:48.078154 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:40:48.132926 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-1b2903a1-02c4-4dae-910e-a0bc3c168427 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:40:48.134125 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-1b2903a1-02c4-4dae-910e-a0bc3c168427 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:40:48.234982 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 715/1401] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 10:40:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 10:40:48.280948 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3970207a-27ab-43a5-a054-4f159918ced2 None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:40:48.282573 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-3970207a-27ab-43a5-a054-4f159918ced2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:40:48.439459 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3970207a-27ab-43a5-a054-4f159918ced2 None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:40:48.473753 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adf9920-3d24-4f4c-908a-e8cd7c79ac98 None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:40:48.474048 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adf9920-3d24-4f4c-908a-e8cd7c79ac98 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:40:48.474325 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adf9920-3d24-4f4c-908a-e8cd7c79ac98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:40:48.494710 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:40:48.543376 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-6adf9920-3d24-4f4c-908a-e8cd7c79ac98 None None] MFA Rules not processed for user `3d9fed24b4ba44b0b5961a603c4362c4`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:40:48.544635 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-6adf9920-3d24-4f4c-908a-e8cd7c79ac98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:40:48.561446 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bac83e-7788-46c4-97fa-79ef81210f5c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e059d8b6c28c4fcba925d05d7c98e4ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:40:48.562071 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bac83e-7788-46c4-97fa-79ef81210f5c service nova] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:40:48.562354 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bac83e-7788-46c4-97fa-79ef81210f5c service nova] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:40:48.562651 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bac83e-7788-46c4-97fa-79ef81210f5c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:40:48.564273 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-76bac83e-7788-46c4-97fa-79ef81210f5c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:40:48.599864 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:40:48.637367 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 716/1402] 10.222.0.36 () {58 vars in 1009 bytes} [Wed Jul 27 10:40:48 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 10:40:48.657907 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bac83e-7788-46c4-97fa-79ef81210f5c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:40:48.659572 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bac83e-7788-46c4-97fa-79ef81210f5c service nova] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:40:48.678183 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 687/1403] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:40:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 10:40:48.682159 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76be38d3-da47-4091-9047-40eb46da3150 None None] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:40:48.682442 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76be38d3-da47-4091-9047-40eb46da3150 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:40:48.682756 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76be38d3-da47-4091-9047-40eb46da3150 None None] PATH_INFO: `/` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:40:48.684497 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 717/1404] 10.222.0.36 () {54 vars in 943 bytes} [Wed Jul 27 10:40:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 10:40:48.691205 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7826809-1142-4532-8e7b-4a42d706be4a None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:40:48.691500 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7826809-1142-4532-8e7b-4a42d706be4a None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:40:48.691788 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7826809-1142-4532-8e7b-4a42d706be4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:40:48.766085 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:40:48.777005 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-c7826809-1142-4532-8e7b-4a42d706be4a None None] MFA Rules not processed for user `89ac506eefcd4e42a84b44fa15fb3975`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:40:48.778244 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-c7826809-1142-4532-8e7b-4a42d706be4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:40:48.895667 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 688/1405] 10.222.0.36 () {58 vars in 1040 bytes} [Wed Jul 27 10:40:48 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 10:40:48.905063 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76be38d3-da47-4091-9047-40eb46da3150 None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:40:48.906678 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-76be38d3-da47-4091-9047-40eb46da3150 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:40:49.061808 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76be38d3-da47-4091-9047-40eb46da3150 None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:40:49.117779 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:40:49.166867 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17dad65-81ab-4f8f-a32f-f7c4d8e830be service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89ac506eefcd4e42a84b44fa15fb3975', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:40:49.167490 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17dad65-81ab-4f8f-a32f-f7c4d8e830be service swift] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:40:49.167767 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17dad65-81ab-4f8f-a32f-f7c4d8e830be service swift] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:40:49.168038 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17dad65-81ab-4f8f-a32f-f7c4d8e830be service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:40:49.169674 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-b17dad65-81ab-4f8f-a32f-f7c4d8e830be service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:40:49.256389 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17dad65-81ab-4f8f-a32f-f7c4d8e830be service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:40:49.258075 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17dad65-81ab-4f8f-a32f-f7c4d8e830be service swift] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:40:49.267179 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 718/1406] 10.222.0.36 () {60 vars in 1384 bytes} [Wed Jul 27 10:40:48 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 27 10:40:50.725624 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bd3032-f726-4300-9c29-337d22e8cbb9 None None] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:40:50.726085 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bd3032-f726-4300-9c29-337d22e8cbb9 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:40:50.726294 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bd3032-f726-4300-9c29-337d22e8cbb9 None None] PATH_INFO: `/` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:40:50.727908 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 689/1407] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 10:40:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 10:40:50.733416 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67adb4e1-7775-4c05-b92e-7eadccf94abd None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:40:50.733829 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67adb4e1-7775-4c05-b92e-7eadccf94abd None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:40:50.734027 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67adb4e1-7775-4c05-b92e-7eadccf94abd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:40:50.807326 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-67adb4e1-7775-4c05-b92e-7eadccf94abd None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:40:50.808539 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-67adb4e1-7775-4c05-b92e-7eadccf94abd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:40:50.854415 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:40:50.922671 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 719/1408] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 10:40:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:40:50.972492 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bd3032-f726-4300-9c29-337d22e8cbb9 None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:40:50.974324 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-24bd3032-f726-4300-9c29-337d22e8cbb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:40:51.102206 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:40:51.155768 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bd3032-f726-4300-9c29-337d22e8cbb9 None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:40:51.290506 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cfee2b-6531-4c27-8cba-00219382846d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e059d8b6c28c4fcba925d05d7c98e4ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:40:51.291303 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cfee2b-6531-4c27-8cba-00219382846d service nova] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:40:51.291590 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cfee2b-6531-4c27-8cba-00219382846d service nova] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:40:51.291866 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cfee2b-6531-4c27-8cba-00219382846d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:40:51.293510 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-51cfee2b-6531-4c27-8cba-00219382846d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:40:51.383196 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:40:51.402196 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51cfee2b-6531-4c27-8cba-00219382846d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:40:51.405395 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51cfee2b-6531-4c27-8cba-00219382846d service nova] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:40:51.432791 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 690/1409] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:40:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 461 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 10:41:00.920764 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=25 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:41:01.430217 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=12 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:41:11.836619 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4e7bdc-49ee-40af-ba73-b914cb5deb39 None None] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:41:11.837193 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4e7bdc-49ee-40af-ba73-b914cb5deb39 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:41:11.837193 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4e7bdc-49ee-40af-ba73-b914cb5deb39 None None] PATH_INFO: `/` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:41:11.838588 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 720/1410] 10.222.0.36 () {54 vars in 918 bytes} [Wed Jul 27 10:41:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 10:41:11.844252 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0c1822-7b1c-4489-a42d-607bc21f539f None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:41:11.844664 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0c1822-7b1c-4489-a42d-607bc21f539f None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:41:11.844790 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0c1822-7b1c-4489-a42d-607bc21f539f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:41:11.920016 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-4a0c1822-7b1c-4489-a42d-607bc21f539f None None] MFA Rules not processed for user `16cc6179e9a04b6398fbacb40ad1c555`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:41:11.921228 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-4a0c1822-7b1c-4489-a42d-607bc21f539f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:41:12.027302 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 691/1411] 10.222.0.36 () {58 vars in 1015 bytes} [Wed Jul 27 10:41:11 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:41:12.041664 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32348116-1dbb-4702-8dc3-c7ce9c888398 None None] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:41:12.041954 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32348116-1dbb-4702-8dc3-c7ce9c888398 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:41:12.042234 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32348116-1dbb-4702-8dc3-c7ce9c888398 None None] PATH_INFO: `/` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:41:12.043660 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 721/1412] 10.222.0.36 () {54 vars in 918 bytes} [Wed Jul 27 10:41:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 10:41:12.048058 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc46d00d-a1fe-440c-a239-be3d07256cd7 None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:41:12.048349 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc46d00d-a1fe-440c-a239-be3d07256cd7 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:41:12.048630 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc46d00d-a1fe-440c-a239-be3d07256cd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:41:12.121406 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:41:12.124535 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-cc46d00d-a1fe-440c-a239-be3d07256cd7 None None] MFA Rules not processed for user `e059d8b6c28c4fcba925d05d7c98e4ac`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:41:12.126239 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-cc46d00d-a1fe-440c-a239-be3d07256cd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:41:12.239215 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 692/1413] 10.222.0.36 () {58 vars in 1015 bytes} [Wed Jul 27 10:41:12 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:41:14.702141 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0be3be-2530-4de6-8be9-f9be59d2b9e9 None None] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:41:14.702727 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0be3be-2530-4de6-8be9-f9be59d2b9e9 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:41:14.702727 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0be3be-2530-4de6-8be9-f9be59d2b9e9 None None] PATH_INFO: `/` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:41:14.704008 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 722/1414] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 27 10:41:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 10:41:14.709137 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f85903-f452-4755-8146-9d64a9702d41 None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:41:14.709563 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f85903-f452-4755-8146-9d64a9702d41 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:41:14.709703 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f85903-f452-4755-8146-9d64a9702d41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:41:14.775625 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-c9f85903-f452-4755-8146-9d64a9702d41 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:41:14.776834 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-c9f85903-f452-4755-8146-9d64a9702d41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:41:14.830774 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:41:14.871355 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 693/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 27 10:41:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:41:14.915312 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0be3be-2530-4de6-8be9-f9be59d2b9e9 None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:41:14.916826 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-bf0be3be-2530-4de6-8be9-f9be59d2b9e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:41:15.061697 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0be3be-2530-4de6-8be9-f9be59d2b9e9 None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:41:15.133486 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:41:15.173385 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba0a7e2-7874-49bd-a795-d8f54ef0a256 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e059d8b6c28c4fcba925d05d7c98e4ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:41:15.174010 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba0a7e2-7874-49bd-a795-d8f54ef0a256 service nova] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:41:15.174293 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba0a7e2-7874-49bd-a795-d8f54ef0a256 service nova] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:41:15.174567 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba0a7e2-7874-49bd-a795-d8f54ef0a256 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:41:15.176227 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-0ba0a7e2-7874-49bd-a795-d8f54ef0a256 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:41:15.264262 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba0a7e2-7874-49bd-a795-d8f54ef0a256 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:41:15.265876 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba0a7e2-7874-49bd-a795-d8f54ef0a256 service nova] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:41:15.283821 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 723/1416] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:41:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 10:41:16.188201 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e26841-d8e3-4d7f-b9e1-4dc18dd4c1a7 None None] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:41:16.188754 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e26841-d8e3-4d7f-b9e1-4dc18dd4c1a7 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:41:16.188754 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e26841-d8e3-4d7f-b9e1-4dc18dd4c1a7 None None] PATH_INFO: `/` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:41:16.190115 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 694/1417] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 27 10:41:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 10:41:16.195342 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717be7da-0828-4045-b542-c1747e375c75 None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:41:16.195740 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717be7da-0828-4045-b542-c1747e375c75 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:41:16.195847 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717be7da-0828-4045-b542-c1747e375c75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:41:16.271934 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-717be7da-0828-4045-b542-c1747e375c75 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:41:16.273149 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-717be7da-0828-4045-b542-c1747e375c75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:41:16.302708 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:41:16.371177 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 724/1418] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 27 10:41:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:41:16.537933 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e26841-d8e3-4d7f-b9e1-4dc18dd4c1a7 None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:41:16.539414 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-a2e26841-d8e3-4d7f-b9e1-4dc18dd4c1a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:41:16.685761 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e26841-d8e3-4d7f-b9e1-4dc18dd4c1a7 None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:41:16.725180 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:41:16.799337 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51584790-fa93-4199-8c28-3b1026d60b6a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e059d8b6c28c4fcba925d05d7c98e4ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:41:16.799964 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51584790-fa93-4199-8c28-3b1026d60b6a service nova] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:41:16.800247 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51584790-fa93-4199-8c28-3b1026d60b6a service nova] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:41:16.800532 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51584790-fa93-4199-8c28-3b1026d60b6a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:41:16.802150 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-51584790-fa93-4199-8c28-3b1026d60b6a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:41:16.890369 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51584790-fa93-4199-8c28-3b1026d60b6a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:41:16.892039 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51584790-fa93-4199-8c28-3b1026d60b6a service nova] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:41:16.909890 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 695/1419] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:41:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 10:41:17.791938 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9430662-f4b8-40eb-be97-e01f83d47bca None None] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:41:17.792787 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9430662-f4b8-40eb-be97-e01f83d47bca None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:41:17.792787 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9430662-f4b8-40eb-be97-e01f83d47bca None None] PATH_INFO: `/` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:41:17.794547 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 725/1420] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 27 10:41:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 10:41:17.800054 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef607fbc-a255-49c5-b06f-bebbbacdd565 None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:41:17.800671 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef607fbc-a255-49c5-b06f-bebbbacdd565 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:41:17.800807 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef607fbc-a255-49c5-b06f-bebbbacdd565 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:41:17.864578 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:41:17.889525 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-ef607fbc-a255-49c5-b06f-bebbbacdd565 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:41:17.890756 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-ef607fbc-a255-49c5-b06f-bebbbacdd565 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:41:17.986828 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 696/1421] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 27 10:41:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:41:18.140865 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9430662-f4b8-40eb-be97-e01f83d47bca None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:41:18.142386 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-e9430662-f4b8-40eb-be97-e01f83d47bca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:41:18.288101 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9430662-f4b8-40eb-be97-e01f83d47bca None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:41:18.300602 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:41:18.401231 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfe2e4c-ffd0-429e-a498-6c87a585db7b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e059d8b6c28c4fcba925d05d7c98e4ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:41:18.401776 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfe2e4c-ffd0-429e-a498-6c87a585db7b service nova] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:41:18.402059 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfe2e4c-ffd0-429e-a498-6c87a585db7b service nova] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:41:18.402334 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfe2e4c-ffd0-429e-a498-6c87a585db7b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:41:18.404538 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-0dfe2e4c-ffd0-429e-a498-6c87a585db7b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:41:18.493177 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfe2e4c-ffd0-429e-a498-6c87a585db7b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:41:18.494806 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfe2e4c-ffd0-429e-a498-6c87a585db7b service nova] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:41:18.512861 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 726/1422] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:41:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 10:41:19.360851 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e555c0-f26a-4161-aba0-ca89083ac4e7 None None] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:41:19.361387 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e555c0-f26a-4161-aba0-ca89083ac4e7 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:41:19.361387 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e555c0-f26a-4161-aba0-ca89083ac4e7 None None] PATH_INFO: `/` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:41:19.362766 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 697/1423] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 27 10:41:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 10:41:19.367589 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a31319-28a7-4f30-9ecb-fa2ceda33e3e None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:41:19.367973 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a31319-28a7-4f30-9ecb-fa2ceda33e3e None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:41:19.368224 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a31319-28a7-4f30-9ecb-fa2ceda33e3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:41:19.389185 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:41:19.445032 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-e2a31319-28a7-4f30-9ecb-fa2ceda33e3e None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:41:19.446224 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-e2a31319-28a7-4f30-9ecb-fa2ceda33e3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:41:19.543447 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 727/1424] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 27 10:41:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:41:19.711421 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e555c0-f26a-4161-aba0-ca89083ac4e7 None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:41:19.712929 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-c9e555c0-f26a-4161-aba0-ca89083ac4e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:41:19.819736 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:41:19.854706 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e555c0-f26a-4161-aba0-ca89083ac4e7 None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:41:19.967048 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edea866-0e88-4f92-8908-8a91ec7677f7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e059d8b6c28c4fcba925d05d7c98e4ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:41:19.967675 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edea866-0e88-4f92-8908-8a91ec7677f7 service nova] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:41:19.967957 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edea866-0e88-4f92-8908-8a91ec7677f7 service nova] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:41:19.968231 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edea866-0e88-4f92-8908-8a91ec7677f7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:41:19.969854 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-4edea866-0e88-4f92-8908-8a91ec7677f7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:41:20.046049 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:41:20.059685 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4edea866-0e88-4f92-8908-8a91ec7677f7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:41:20.061298 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4edea866-0e88-4f92-8908-8a91ec7677f7 service nova] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:41:20.078960 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 698/1425] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:41:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 10:41:20.938018 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b212faa9-5828-4a31-a189-2ae9a8fa9941 None None] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:41:20.938571 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b212faa9-5828-4a31-a189-2ae9a8fa9941 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:41:20.938696 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b212faa9-5828-4a31-a189-2ae9a8fa9941 None None] PATH_INFO: `/` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:41:20.940153 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 728/1426] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 27 10:41:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 10:41:20.945158 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cc29ad-6165-4d3b-aa9b-e9018d3746b4 None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:41:20.945587 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cc29ad-6165-4d3b-aa9b-e9018d3746b4 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:41:20.945725 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cc29ad-6165-4d3b-aa9b-e9018d3746b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:41:21.021419 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-63cc29ad-6165-4d3b-aa9b-e9018d3746b4 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:41:21.022579 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-63cc29ad-6165-4d3b-aa9b-e9018d3746b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:41:21.117194 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 699/1427] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 27 10:41:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:41:21.267464 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b212faa9-5828-4a31-a189-2ae9a8fa9941 None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:41:21.268970 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-b212faa9-5828-4a31-a189-2ae9a8fa9941 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:41:21.351421 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:41:21.414593 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b212faa9-5828-4a31-a189-2ae9a8fa9941 None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:41:21.525011 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffcfe9f-50d3-4b88-8e09-e68dd73d14d7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e059d8b6c28c4fcba925d05d7c98e4ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:41:21.525640 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffcfe9f-50d3-4b88-8e09-e68dd73d14d7 service nova] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:41:21.525923 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffcfe9f-50d3-4b88-8e09-e68dd73d14d7 service nova] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:41:21.526201 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffcfe9f-50d3-4b88-8e09-e68dd73d14d7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:41:21.527864 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-1ffcfe9f-50d3-4b88-8e09-e68dd73d14d7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:41:21.575529 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:41:21.616677 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffcfe9f-50d3-4b88-8e09-e68dd73d14d7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:41:21.618195 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffcfe9f-50d3-4b88-8e09-e68dd73d14d7 service nova] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:41:21.636252 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 729/1428] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:41:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 10:41:22.445929 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df76424-179c-4c42-af29-d8894dc481ec None None] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:41:22.446521 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df76424-179c-4c42-af29-d8894dc481ec None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:41:22.446521 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df76424-179c-4c42-af29-d8894dc481ec None None] PATH_INFO: `/` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:41:22.447846 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 700/1429] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 27 10:41:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 10:41:22.453360 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57266546-d9cb-4c43-aecb-d76b6f27a150 None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:41:22.453780 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57266546-d9cb-4c43-aecb-d76b6f27a150 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:41:22.453913 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57266546-d9cb-4c43-aecb-d76b6f27a150 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:41:22.529355 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-57266546-d9cb-4c43-aecb-d76b6f27a150 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:41:22.530566 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-57266546-d9cb-4c43-aecb-d76b6f27a150 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:41:22.624816 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 730/1430] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 27 10:41:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:41:22.779328 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df76424-179c-4c42-af29-d8894dc481ec None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:41:22.780780 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-1df76424-179c-4c42-af29-d8894dc481ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:41:22.830264 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:41:22.925970 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df76424-179c-4c42-af29-d8894dc481ec None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:41:23.038927 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddc075d-fa7b-4a11-b375-4c91e2001680 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e059d8b6c28c4fcba925d05d7c98e4ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:41:23.039482 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddc075d-fa7b-4a11-b375-4c91e2001680 service nova] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:41:23.039772 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddc075d-fa7b-4a11-b375-4c91e2001680 service nova] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:41:23.040047 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddc075d-fa7b-4a11-b375-4c91e2001680 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:41:23.041703 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-eddc075d-fa7b-4a11-b375-4c91e2001680 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:41:23.048516 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:41:23.130411 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddc075d-fa7b-4a11-b375-4c91e2001680 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:41:23.132299 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddc075d-fa7b-4a11-b375-4c91e2001680 service nova] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:41:23.150377 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 701/1431] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:41:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 10:41:23.958233 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc5e8c8-dbf4-4bf2-bb9a-6c487030384d None None] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:41:23.958791 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc5e8c8-dbf4-4bf2-bb9a-6c487030384d None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:41:23.958916 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc5e8c8-dbf4-4bf2-bb9a-6c487030384d None None] PATH_INFO: `/` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:41:23.960219 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 731/1432] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 27 10:41:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 10:41:23.965312 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a64227-8e62-4ab7-bf7c-15b2d2f937c8 None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:41:23.965731 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a64227-8e62-4ab7-bf7c-15b2d2f937c8 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:41:23.965863 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a64227-8e62-4ab7-bf7c-15b2d2f937c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:41:24.040803 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-40a64227-8e62-4ab7-bf7c-15b2d2f937c8 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:41:24.041937 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-40a64227-8e62-4ab7-bf7c-15b2d2f937c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:41:24.108711 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:41:24.139495 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 702/1433] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 27 10:41:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:41:24.290559 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc5e8c8-dbf4-4bf2-bb9a-6c487030384d None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:41:24.292152 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-2bc5e8c8-dbf4-4bf2-bb9a-6c487030384d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:41:24.310390 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:41:24.442070 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc5e8c8-dbf4-4bf2-bb9a-6c487030384d None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:41:24.528204 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:41:24.554604 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde98e6d-20a9-4100-b9a9-5e0dc453c9b3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e059d8b6c28c4fcba925d05d7c98e4ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:41:24.555243 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde98e6d-20a9-4100-b9a9-5e0dc453c9b3 service nova] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:41:24.555520 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde98e6d-20a9-4100-b9a9-5e0dc453c9b3 service nova] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:41:24.555791 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde98e6d-20a9-4100-b9a9-5e0dc453c9b3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:41:24.557392 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-dde98e6d-20a9-4100-b9a9-5e0dc453c9b3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:41:24.649823 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde98e6d-20a9-4100-b9a9-5e0dc453c9b3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:41:24.651487 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde98e6d-20a9-4100-b9a9-5e0dc453c9b3 service nova] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:41:24.671003 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 732/1434] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:41:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 10:41:25.480509 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c688a778-eef7-4df9-a318-3884f6521480 None None] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:41:25.480980 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c688a778-eef7-4df9-a318-3884f6521480 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:41:25.481088 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c688a778-eef7-4df9-a318-3884f6521480 None None] PATH_INFO: `/` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:41:25.482448 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 703/1435] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 27 10:41:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 10:41:25.487620 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35111538-a303-42e5-87e8-fcffa227deba None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:41:25.488071 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35111538-a303-42e5-87e8-fcffa227deba None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:41:25.488177 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35111538-a303-42e5-87e8-fcffa227deba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:41:25.566295 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-35111538-a303-42e5-87e8-fcffa227deba None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:41:25.567526 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-35111538-a303-42e5-87e8-fcffa227deba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:41:25.608272 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:41:25.665807 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 733/1436] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 27 10:41:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 10:41:25.815612 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c688a778-eef7-4df9-a318-3884f6521480 None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:41:25.817223 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-c688a778-eef7-4df9-a318-3884f6521480 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:41:25.960657 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c688a778-eef7-4df9-a318-3884f6521480 None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:41:26.016477 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:41:26.076006 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e3ed04-16db-454d-8391-040cc2764aba service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e059d8b6c28c4fcba925d05d7c98e4ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:41:26.076566 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e3ed04-16db-454d-8391-040cc2764aba service nova] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:41:26.076849 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e3ed04-16db-454d-8391-040cc2764aba service nova] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:41:26.077126 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e3ed04-16db-454d-8391-040cc2764aba service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:41:26.078785 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-b7e3ed04-16db-454d-8391-040cc2764aba service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:41:26.166916 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e3ed04-16db-454d-8391-040cc2764aba service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:41:26.168547 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e3ed04-16db-454d-8391-040cc2764aba service nova] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:41:26.186698 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 704/1437] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:41:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 10:41:26.988105 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c862243-65ed-4eb8-9eb4-75062cc9b085 None None] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:41:26.988581 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c862243-65ed-4eb8-9eb4-75062cc9b085 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:41:26.988695 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c862243-65ed-4eb8-9eb4-75062cc9b085 None None] PATH_INFO: `/` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:41:26.990082 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 734/1438] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 27 10:41:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 10:41:26.995000 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ab8cd1-2357-41b5-b302-6f4441db16a0 None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:41:26.995425 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ab8cd1-2357-41b5-b302-6f4441db16a0 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:41:26.995555 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ab8cd1-2357-41b5-b302-6f4441db16a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:41:27.068937 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-30ab8cd1-2357-41b5-b302-6f4441db16a0 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:41:27.070125 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-30ab8cd1-2357-41b5-b302-6f4441db16a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:41:27.072172 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:41:27.164765 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 705/1439] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 27 10:41:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:41:27.314050 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c862243-65ed-4eb8-9eb4-75062cc9b085 None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:41:27.315617 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-7c862243-65ed-4eb8-9eb4-75062cc9b085 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:41:27.462515 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c862243-65ed-4eb8-9eb4-75062cc9b085 None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:41:27.491078 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:41:27.575859 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6504be99-6358-4d1a-adfe-e2f7e3fb279c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e059d8b6c28c4fcba925d05d7c98e4ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:41:27.576470 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6504be99-6358-4d1a-adfe-e2f7e3fb279c service nova] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:41:27.576748 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6504be99-6358-4d1a-adfe-e2f7e3fb279c service nova] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:41:27.577019 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6504be99-6358-4d1a-adfe-e2f7e3fb279c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:41:27.578652 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-6504be99-6358-4d1a-adfe-e2f7e3fb279c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:41:27.665437 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6504be99-6358-4d1a-adfe-e2f7e3fb279c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:41:27.667045 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6504be99-6358-4d1a-adfe-e2f7e3fb279c service nova] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:41:27.685154 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 735/1440] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:41:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 10:41:29.589030 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86d790b-5b38-4670-92c5-e518296279a5 None None] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:41:29.589593 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86d790b-5b38-4670-92c5-e518296279a5 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:41:29.589593 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86d790b-5b38-4670-92c5-e518296279a5 None None] PATH_INFO: `/` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:41:29.591566 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 706/1441] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 27 10:41:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 10:41:29.596869 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b3e45e-f573-4d1c-b117-c42302dadcc0 None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:41:29.597256 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b3e45e-f573-4d1c-b117-c42302dadcc0 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:41:29.597438 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b3e45e-f573-4d1c-b117-c42302dadcc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:41:29.640912 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:41:29.668960 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-c7b3e45e-f573-4d1c-b117-c42302dadcc0 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:41:29.670196 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-c7b3e45e-f573-4d1c-b117-c42302dadcc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:41:29.767905 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 736/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 27 10:41:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:41:29.805515 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86d790b-5b38-4670-92c5-e518296279a5 None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:41:29.807076 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-d86d790b-5b38-4670-92c5-e518296279a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:41:29.909901 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86d790b-5b38-4670-92c5-e518296279a5 None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:41:29.941738 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:41:29.983889 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e8c7e6-e9b7-448d-a282-76975a224ab4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7922d0a1a7c4e808d0b2a80dca0da40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:41:29.984514 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e8c7e6-e9b7-448d-a282-76975a224ab4 service neutron] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:41:29.984797 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e8c7e6-e9b7-448d-a282-76975a224ab4 service neutron] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:41:29.985099 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e8c7e6-e9b7-448d-a282-76975a224ab4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:41:29.986817 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-c0e8c7e6-e9b7-448d-a282-76975a224ab4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:41:30.098327 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e8c7e6-e9b7-448d-a282-76975a224ab4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:41:30.099796 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e8c7e6-e9b7-448d-a282-76975a224ab4 service neutron] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:41:30.118324 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 707/1443] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:41:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 10:41:32.296546 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e23f61-523b-42ea-83b6-9a47f79b6664 None None] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:41:32.297074 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e23f61-523b-42ea-83b6-9a47f79b6664 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:41:32.297074 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e23f61-523b-42ea-83b6-9a47f79b6664 None None] PATH_INFO: `/` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:41:32.298394 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 737/1444] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 27 10:41:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 10:41:32.303708 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2c92e9-c03c-43c6-b2c3-738bca7ee8bb None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:41:32.304156 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2c92e9-c03c-43c6-b2c3-738bca7ee8bb None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:41:32.304265 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2c92e9-c03c-43c6-b2c3-738bca7ee8bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:41:32.368970 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-3a2c92e9-c03c-43c6-b2c3-738bca7ee8bb None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:41:32.370185 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-3a2c92e9-c03c-43c6-b2c3-738bca7ee8bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:41:32.383899 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:41:32.468019 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 708/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 27 10:41:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:41:32.507396 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e23f61-523b-42ea-83b6-9a47f79b6664 None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:41:32.508911 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-e4e23f61-523b-42ea-83b6-9a47f79b6664 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:41:32.609681 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e23f61-523b-42ea-83b6-9a47f79b6664 None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:41:32.621876 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:41:32.684895 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ef79a7-eeab-4012-b782-373ec9d0d401 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7922d0a1a7c4e808d0b2a80dca0da40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:41:32.685516 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ef79a7-eeab-4012-b782-373ec9d0d401 service neutron] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:41:32.685800 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ef79a7-eeab-4012-b782-373ec9d0d401 service neutron] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:41:32.686078 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ef79a7-eeab-4012-b782-373ec9d0d401 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:41:32.687747 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-57ef79a7-eeab-4012-b782-373ec9d0d401 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:41:32.797884 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ef79a7-eeab-4012-b782-373ec9d0d401 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:41:32.799443 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ef79a7-eeab-4012-b782-373ec9d0d401 service neutron] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:41:32.817456 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 738/1446] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:41:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 10:41:35.138243 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d2e4d4-b0be-46b4-8521-57864c378faa None None] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:41:35.138801 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d2e4d4-b0be-46b4-8521-57864c378faa None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:41:35.138801 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d2e4d4-b0be-46b4-8521-57864c378faa None None] PATH_INFO: `/` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:41:35.140160 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 709/1447] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 27 10:41:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 10:41:35.145488 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3120db08-cd0c-4283-893b-31d6b8164cc5 None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:41:35.145902 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3120db08-cd0c-4283-893b-31d6b8164cc5 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:41:35.146033 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3120db08-cd0c-4283-893b-31d6b8164cc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:41:35.195968 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:41:35.214088 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-3120db08-cd0c-4283-893b-31d6b8164cc5 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:41:35.215321 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-3120db08-cd0c-4283-893b-31d6b8164cc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:41:35.312564 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 739/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 27 10:41:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:41:35.350333 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d2e4d4-b0be-46b4-8521-57864c378faa None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:41:35.351980 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-48d2e4d4-b0be-46b4-8521-57864c378faa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:41:35.456025 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d2e4d4-b0be-46b4-8521-57864c378faa None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:41:35.495700 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:41:35.529594 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46296255-15fc-46d6-a9c5-a2d43b2e8828 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7922d0a1a7c4e808d0b2a80dca0da40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:41:35.530209 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46296255-15fc-46d6-a9c5-a2d43b2e8828 service neutron] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:41:35.530488 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46296255-15fc-46d6-a9c5-a2d43b2e8828 service neutron] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:41:35.530792 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46296255-15fc-46d6-a9c5-a2d43b2e8828 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:41:35.532398 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-46296255-15fc-46d6-a9c5-a2d43b2e8828 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:41:35.640910 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46296255-15fc-46d6-a9c5-a2d43b2e8828 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:41:35.642799 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46296255-15fc-46d6-a9c5-a2d43b2e8828 service neutron] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:41:35.783728 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 710/1449] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:41:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 10:41:37.922530 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9b56d8-6c36-48bb-8606-b05bfce1fccf None None] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:41:37.923119 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9b56d8-6c36-48bb-8606-b05bfce1fccf None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:41:37.923119 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9b56d8-6c36-48bb-8606-b05bfce1fccf None None] PATH_INFO: `/` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:41:37.924484 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 740/1450] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 27 10:41:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 10:41:37.930445 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970a7a75-0a62-4a9c-92c4-a5f8b1d88496 None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:41:37.930905 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970a7a75-0a62-4a9c-92c4-a5f8b1d88496 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:41:37.931021 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970a7a75-0a62-4a9c-92c4-a5f8b1d88496 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:41:37.997910 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-970a7a75-0a62-4a9c-92c4-a5f8b1d88496 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:41:37.999052 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-970a7a75-0a62-4a9c-92c4-a5f8b1d88496 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:41:38.019373 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:41:38.096046 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 711/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 27 10:41:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:41:38.135426 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9b56d8-6c36-48bb-8606-b05bfce1fccf None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:41:38.136977 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-fa9b56d8-6c36-48bb-8606-b05bfce1fccf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:41:38.357594 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9b56d8-6c36-48bb-8606-b05bfce1fccf None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:41:38.376751 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:41:38.429057 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d5476c-8438-4a56-919e-8c20d5a0858d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7922d0a1a7c4e808d0b2a80dca0da40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:41:38.429669 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d5476c-8438-4a56-919e-8c20d5a0858d service neutron] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:41:38.429944 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d5476c-8438-4a56-919e-8c20d5a0858d service neutron] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:41:38.430217 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d5476c-8438-4a56-919e-8c20d5a0858d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:41:38.431847 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-60d5476c-8438-4a56-919e-8c20d5a0858d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:41:38.538976 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d5476c-8438-4a56-919e-8c20d5a0858d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:41:38.540420 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d5476c-8438-4a56-919e-8c20d5a0858d service neutron] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:41:38.558225 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 741/1452] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:41:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 10:41:40.597562 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8729665-6324-4f3d-af3e-99121a4e7c8b None None] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:41:40.598147 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8729665-6324-4f3d-af3e-99121a4e7c8b None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:41:40.598147 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8729665-6324-4f3d-af3e-99121a4e7c8b None None] PATH_INFO: `/` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:41:40.599545 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 712/1453] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 27 10:41:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 10:41:40.604911 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd75f7b5-535a-4616-a2d6-f96cfd712e0a None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:41:40.605315 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd75f7b5-535a-4616-a2d6-f96cfd712e0a None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:41:40.605494 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd75f7b5-535a-4616-a2d6-f96cfd712e0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:41:40.668699 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:41:40.672328 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-fd75f7b5-535a-4616-a2d6-f96cfd712e0a None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:41:40.673711 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-fd75f7b5-535a-4616-a2d6-f96cfd712e0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:41:40.771486 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 742/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 27 10:41:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:41:40.811183 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8729665-6324-4f3d-af3e-99121a4e7c8b None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:41:40.812715 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-f8729665-6324-4f3d-af3e-99121a4e7c8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:41:40.917618 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8729665-6324-4f3d-af3e-99121a4e7c8b None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:41:40.966989 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:41:40.991805 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a5a5b1-abdf-41ad-baa1-7550e04748f9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7922d0a1a7c4e808d0b2a80dca0da40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:41:40.992428 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a5a5b1-abdf-41ad-baa1-7550e04748f9 service neutron] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:41:40.992711 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a5a5b1-abdf-41ad-baa1-7550e04748f9 service neutron] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:41:40.992985 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a5a5b1-abdf-41ad-baa1-7550e04748f9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:41:40.994609 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-56a5a5b1-abdf-41ad-baa1-7550e04748f9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:41:41.101945 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a5a5b1-abdf-41ad-baa1-7550e04748f9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:41:41.103391 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a5a5b1-abdf-41ad-baa1-7550e04748f9 service neutron] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:41:41.121391 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 713/1455] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:41:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 10:41:43.180485 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4dddd9-3361-44c6-87e1-9389cdbcf18f None None] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:41:43.181043 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4dddd9-3361-44c6-87e1-9389cdbcf18f None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:41:43.181184 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4dddd9-3361-44c6-87e1-9389cdbcf18f None None] PATH_INFO: `/` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:41:43.182428 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 743/1456] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 27 10:41:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 10:41:43.188165 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0100ab89-d977-4451-a036-68e55a25a2e0 None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:41:43.188545 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0100ab89-d977-4451-a036-68e55a25a2e0 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:41:43.188722 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0100ab89-d977-4451-a036-68e55a25a2e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:41:43.253439 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-0100ab89-d977-4451-a036-68e55a25a2e0 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:41:43.254567 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-0100ab89-d977-4451-a036-68e55a25a2e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:41:43.284035 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:41:43.352199 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 714/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 27 10:41:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:41:43.391043 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4dddd9-3361-44c6-87e1-9389cdbcf18f None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:41:43.392589 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-3b4dddd9-3361-44c6-87e1-9389cdbcf18f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:41:43.497116 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4dddd9-3361-44c6-87e1-9389cdbcf18f None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:41:43.525818 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:41:43.571195 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fa9d90-b5fe-4474-8ec3-126095130f4f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7922d0a1a7c4e808d0b2a80dca0da40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:41:43.571806 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fa9d90-b5fe-4474-8ec3-126095130f4f service neutron] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:41:43.572084 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fa9d90-b5fe-4474-8ec3-126095130f4f service neutron] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:41:43.572355 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fa9d90-b5fe-4474-8ec3-126095130f4f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:41:43.573964 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-79fa9d90-b5fe-4474-8ec3-126095130f4f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:41:43.681654 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79fa9d90-b5fe-4474-8ec3-126095130f4f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:41:43.683146 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79fa9d90-b5fe-4474-8ec3-126095130f4f service neutron] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:41:43.701632 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 744/1458] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:41:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 10:41:45.746879 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb7f6a3-46ac-4e6c-8199-4ffccd1c91b8 None None] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:41:45.747450 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb7f6a3-46ac-4e6c-8199-4ffccd1c91b8 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:41:45.747450 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb7f6a3-46ac-4e6c-8199-4ffccd1c91b8 None None] PATH_INFO: `/` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:41:45.748779 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 715/1459] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 27 10:41:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 10:41:45.753990 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc899ed9-54c0-4442-a3bb-0156cc32d7a1 None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:41:45.754389 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc899ed9-54c0-4442-a3bb-0156cc32d7a1 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:41:45.754542 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc899ed9-54c0-4442-a3bb-0156cc32d7a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:41:45.818802 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-bc899ed9-54c0-4442-a3bb-0156cc32d7a1 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:41:45.819986 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-bc899ed9-54c0-4442-a3bb-0156cc32d7a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:41:45.828514 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:41:45.914661 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 745/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 27 10:41:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:41:45.951978 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb7f6a3-46ac-4e6c-8199-4ffccd1c91b8 None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:41:45.953488 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-ddb7f6a3-46ac-4e6c-8199-4ffccd1c91b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:41:46.055183 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb7f6a3-46ac-4e6c-8199-4ffccd1c91b8 None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:41:46.109190 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:41:46.126956 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82450188-7c38-41ec-8f84-6cefe1c4a0e6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7922d0a1a7c4e808d0b2a80dca0da40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:41:46.127567 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82450188-7c38-41ec-8f84-6cefe1c4a0e6 service neutron] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:41:46.127872 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82450188-7c38-41ec-8f84-6cefe1c4a0e6 service neutron] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:41:46.128142 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82450188-7c38-41ec-8f84-6cefe1c4a0e6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:41:46.129743 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-82450188-7c38-41ec-8f84-6cefe1c4a0e6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:41:46.236085 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82450188-7c38-41ec-8f84-6cefe1c4a0e6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:41:46.237522 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82450188-7c38-41ec-8f84-6cefe1c4a0e6 service neutron] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:41:46.255855 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 716/1461] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:41:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 10:41:48.350082 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acaa5e09-8ee7-4a45-a428-0808fc982b6f None None] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:41:48.350669 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acaa5e09-8ee7-4a45-a428-0808fc982b6f None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:41:48.350669 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acaa5e09-8ee7-4a45-a428-0808fc982b6f None None] PATH_INFO: `/` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:41:48.352204 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 746/1462] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 27 10:41:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 10:41:48.357312 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36c9603-1eb3-4c4f-8115-65e839d09e47 None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:41:48.357773 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36c9603-1eb3-4c4f-8115-65e839d09e47 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:41:48.357872 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36c9603-1eb3-4c4f-8115-65e839d09e47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:41:48.424528 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-d36c9603-1eb3-4c4f-8115-65e839d09e47 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:41:48.425751 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-d36c9603-1eb3-4c4f-8115-65e839d09e47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:41:48.464437 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:41:48.522760 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 717/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 27 10:41:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:41:48.560309 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acaa5e09-8ee7-4a45-a428-0808fc982b6f None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:41:48.561847 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-acaa5e09-8ee7-4a45-a428-0808fc982b6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:41:48.665742 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acaa5e09-8ee7-4a45-a428-0808fc982b6f None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:41:48.702804 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:41:48.738265 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e84fb24-c3a5-4d06-8f8f-b558afb6f2a6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7922d0a1a7c4e808d0b2a80dca0da40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:41:48.738913 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e84fb24-c3a5-4d06-8f8f-b558afb6f2a6 service neutron] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:41:48.739191 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e84fb24-c3a5-4d06-8f8f-b558afb6f2a6 service neutron] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:41:48.739462 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e84fb24-c3a5-4d06-8f8f-b558afb6f2a6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:41:48.741069 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-0e84fb24-c3a5-4d06-8f8f-b558afb6f2a6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:41:48.847282 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e84fb24-c3a5-4d06-8f8f-b558afb6f2a6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:41:48.848744 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e84fb24-c3a5-4d06-8f8f-b558afb6f2a6 service neutron] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:41:48.867631 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 747/1464] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:41:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 10:41:50.864613 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3239637e-761a-4df0-901f-a8ffba481f33 None None] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:41:50.865139 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3239637e-761a-4df0-901f-a8ffba481f33 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:41:50.865139 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3239637e-761a-4df0-901f-a8ffba481f33 None None] PATH_INFO: `/` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:41:50.866470 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 718/1465] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 27 10:41:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 10:41:50.871673 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac35b955-b21c-46ed-8837-f7f94942603d None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:41:50.872078 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac35b955-b21c-46ed-8837-f7f94942603d None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:41:50.872239 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac35b955-b21c-46ed-8837-f7f94942603d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:41:50.939732 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-ac35b955-b21c-46ed-8837-f7f94942603d None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:41:50.941427 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-ac35b955-b21c-46ed-8837-f7f94942603d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:41:50.960188 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:41:51.039981 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 748/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 27 10:41:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:41:51.076800 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3239637e-761a-4df0-901f-a8ffba481f33 None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:41:51.078303 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-3239637e-761a-4df0-901f-a8ffba481f33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:41:51.178097 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3239637e-761a-4df0-901f-a8ffba481f33 None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:41:51.242395 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:41:51.249757 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0821f54-b2d9-409f-ae03-19825fef5571 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7922d0a1a7c4e808d0b2a80dca0da40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:41:51.250375 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0821f54-b2d9-409f-ae03-19825fef5571 service neutron] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:41:51.250699 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0821f54-b2d9-409f-ae03-19825fef5571 service neutron] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:41:51.250977 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0821f54-b2d9-409f-ae03-19825fef5571 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:41:51.252597 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-a0821f54-b2d9-409f-ae03-19825fef5571 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:41:51.363055 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0821f54-b2d9-409f-ae03-19825fef5571 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:41:51.364499 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0821f54-b2d9-409f-ae03-19825fef5571 service neutron] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:41:51.383302 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 719/1467] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:41:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 10:41:53.370618 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a252966-a1ab-4bcb-9353-c11543f46b35 None None] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:41:53.371154 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a252966-a1ab-4bcb-9353-c11543f46b35 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:41:53.371154 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a252966-a1ab-4bcb-9353-c11543f46b35 None None] PATH_INFO: `/` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:41:53.373102 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 749/1468] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 27 10:41:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 10:41:53.378282 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4779c05e-1fdf-46e7-8cc9-7abe0f6bcf5a None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:41:53.378747 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4779c05e-1fdf-46e7-8cc9-7abe0f6bcf5a None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:41:53.378868 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4779c05e-1fdf-46e7-8cc9-7abe0f6bcf5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:41:53.444533 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-4779c05e-1fdf-46e7-8cc9-7abe0f6bcf5a None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:41:53.445721 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-4779c05e-1fdf-46e7-8cc9-7abe0f6bcf5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:41:53.492266 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:41:53.541273 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 720/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 27 10:41:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:41:53.578065 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a252966-a1ab-4bcb-9353-c11543f46b35 None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:41:53.579657 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-7a252966-a1ab-4bcb-9353-c11543f46b35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:41:53.680662 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a252966-a1ab-4bcb-9353-c11543f46b35 None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:41:53.726670 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:41:53.752661 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d5aced-899b-440a-a2f8-a9ad47e6ba1f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7922d0a1a7c4e808d0b2a80dca0da40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:41:53.753275 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d5aced-899b-440a-a2f8-a9ad47e6ba1f service neutron] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:41:53.753555 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d5aced-899b-440a-a2f8-a9ad47e6ba1f service neutron] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:41:53.753827 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d5aced-899b-440a-a2f8-a9ad47e6ba1f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:41:53.755735 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-66d5aced-899b-440a-a2f8-a9ad47e6ba1f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:41:53.863084 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d5aced-899b-440a-a2f8-a9ad47e6ba1f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:41:53.864512 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d5aced-899b-440a-a2f8-a9ad47e6ba1f service neutron] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:41:53.882515 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 750/1470] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:41:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 10:42:03.539474 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=22 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:42:03.879757 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=66 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:42:13.477948 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a87c70-275f-44e9-b1c2-20462d7cff7b None None] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:42:13.478502 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a87c70-275f-44e9-b1c2-20462d7cff7b None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:42:13.478502 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a87c70-275f-44e9-b1c2-20462d7cff7b None None] PATH_INFO: `/` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:42:13.479872 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 721/1471] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 10:42:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 10:42:13.485184 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301fb247-37e2-423e-bb13-8d3da333d67d None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:42:13.485627 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301fb247-37e2-423e-bb13-8d3da333d67d None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:42:13.485766 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301fb247-37e2-423e-bb13-8d3da333d67d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:42:13.560633 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-301fb247-37e2-423e-bb13-8d3da333d67d None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:42:13.561863 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-301fb247-37e2-423e-bb13-8d3da333d67d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:42:13.659314 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 751/1472] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 10:42:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:42:13.742160 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a68e896-4636-481c-a8ca-7a6cbc70cbd1 None None] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:42:13.742447 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a68e896-4636-481c-a8ca-7a6cbc70cbd1 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:42:13.742757 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a68e896-4636-481c-a8ca-7a6cbc70cbd1 None None] PATH_INFO: `/` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:42:13.744109 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 722/1473] 10.222.0.36 () {54 vars in 966 bytes} [Wed Jul 27 10:42:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 10:42:13.748936 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525bc4da-bed4-4134-85a0-6fe6d160c4d6 None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:42:13.749249 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525bc4da-bed4-4134-85a0-6fe6d160c4d6 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:42:13.749545 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525bc4da-bed4-4134-85a0-6fe6d160c4d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:42:13.825534 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-525bc4da-bed4-4134-85a0-6fe6d160c4d6 None None] MFA Rules not processed for user `bc03c5d232654878aa1ddce15f49fdaa`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:42:13.826683 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-525bc4da-bed4-4134-85a0-6fe6d160c4d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:42:13.846866 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:42:13.904349 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 752/1474] 10.222.0.36 () {58 vars in 1063 bytes} [Wed Jul 27 10:42:13 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:42:13.911950 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a68e896-4636-481c-a8ca-7a6cbc70cbd1 None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:42:13.913515 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-8a68e896-4636-481c-a8ca-7a6cbc70cbd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:42:14.019194 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a68e896-4636-481c-a8ca-7a6cbc70cbd1 None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:42:14.090964 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24bed89-2407-4228-9283-60f347b60965 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc03c5d232654878aa1ddce15f49fdaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:42:14.091509 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24bed89-2407-4228-9283-60f347b60965 service glance] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:42:14.091788 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24bed89-2407-4228-9283-60f347b60965 service glance] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:42:14.092059 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24bed89-2407-4228-9283-60f347b60965 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:42:14.093699 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-a24bed89-2407-4228-9283-60f347b60965 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:42:14.148513 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:42:14.202751 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24bed89-2407-4228-9283-60f347b60965 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:42:14.204188 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24bed89-2407-4228-9283-60f347b60965 service glance] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:42:14.222203 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 723/1475] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:42:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 10:42:16.298445 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1ef65e-6aae-45fc-a00a-59e2d1582266 None None] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:42:16.298996 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1ef65e-6aae-45fc-a00a-59e2d1582266 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:42:16.298996 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1ef65e-6aae-45fc-a00a-59e2d1582266 None None] PATH_INFO: `/` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:42:16.300373 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 753/1476] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 10:42:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 10:42:16.305619 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6932780-5f0c-4f3d-a8f6-c0dc0df6bbc1 None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:42:16.306044 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6932780-5f0c-4f3d-a8f6-c0dc0df6bbc1 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:42:16.306174 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6932780-5f0c-4f3d-a8f6-c0dc0df6bbc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:42:16.372203 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-a6932780-5f0c-4f3d-a8f6-c0dc0df6bbc1 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:42:16.373416 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-a6932780-5f0c-4f3d-a8f6-c0dc0df6bbc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:42:16.468594 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 724/1477] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 10:42:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:42:16.490536 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1ef65e-6aae-45fc-a00a-59e2d1582266 None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:42:16.492122 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-8e1ef65e-6aae-45fc-a00a-59e2d1582266 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:42:16.618935 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1ef65e-6aae-45fc-a00a-59e2d1582266 None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:42:16.671891 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:42:16.719461 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131c79cd-5327-4b4a-a530-b923e0f2ecb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:42:16.720082 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131c79cd-5327-4b4a-a530-b923e0f2ecb1 admin admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:42:16.720366 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131c79cd-5327-4b4a-a530-b923e0f2ecb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:42:16.720647 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131c79cd-5327-4b4a-a530-b923e0f2ecb1 admin admin] PATH_INFO: `/v3/projects` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:42:16.721346 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131c79cd-5327-4b4a-a530-b923e0f2ecb1 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:42:16.723117 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: WARNING py.warnings [None req-131c79cd-5327-4b4a-a530-b923e0f2ecb1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 27 10:42:16.723117 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: warnings.warn(msg) Jul 27 10:42:16.723117 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: Jul 27 10:42:16.724260 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131c79cd-5327-4b4a-a530-b923e0f2ecb1 admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:42:16.742693 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 754/1478] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 10:42:16 2022] GET /identity/v3/projects => generated 2553 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 10:42:18.773931 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee10d0a-a490-48ee-b818-e2daf3cbcf18 None None] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:42:18.774509 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee10d0a-a490-48ee-b818-e2daf3cbcf18 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:42:18.774509 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee10d0a-a490-48ee-b818-e2daf3cbcf18 None None] PATH_INFO: `/` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:42:18.775890 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 725/1479] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 10:42:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 10:42:18.781129 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7137f6af-694e-4d71-8249-ccf7cfabba9e None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:42:18.781567 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7137f6af-694e-4d71-8249-ccf7cfabba9e None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:42:18.781679 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7137f6af-694e-4d71-8249-ccf7cfabba9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:42:18.846301 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-7137f6af-694e-4d71-8249-ccf7cfabba9e None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:42:18.847494 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-7137f6af-694e-4d71-8249-ccf7cfabba9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:42:18.949358 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 755/1480] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 10:42:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:42:18.992578 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee10d0a-a490-48ee-b818-e2daf3cbcf18 None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:42:18.994312 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-6ee10d0a-a490-48ee-b818-e2daf3cbcf18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:42:19.005370 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:42:19.143074 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee10d0a-a490-48ee-b818-e2daf3cbcf18 None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:42:19.221701 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:42:19.255670 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63f0c7a-baad-4003-a7cd-bb3d1c94d508 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e059d8b6c28c4fcba925d05d7c98e4ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:42:19.256291 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63f0c7a-baad-4003-a7cd-bb3d1c94d508 service nova] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:42:19.256570 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63f0c7a-baad-4003-a7cd-bb3d1c94d508 service nova] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:42:19.256841 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63f0c7a-baad-4003-a7cd-bb3d1c94d508 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:42:19.258459 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-e63f0c7a-baad-4003-a7cd-bb3d1c94d508 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:42:19.350208 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63f0c7a-baad-4003-a7cd-bb3d1c94d508 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:42:19.351908 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63f0c7a-baad-4003-a7cd-bb3d1c94d508 service nova] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:42:19.369544 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 726/1481] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:42:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 10:42:21.549091 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36df62af-c28d-4ef8-bb47-633d2e721bd6 None None] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:42:21.549616 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36df62af-c28d-4ef8-bb47-633d2e721bd6 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:42:21.549718 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36df62af-c28d-4ef8-bb47-633d2e721bd6 None None] PATH_INFO: `/` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:42:21.551058 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 756/1482] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 10:42:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 10:42:21.556298 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd784b4-bf52-4a9a-9841-f54aa47fb11d None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:42:21.556705 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd784b4-bf52-4a9a-9841-f54aa47fb11d None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:42:21.556864 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd784b4-bf52-4a9a-9841-f54aa47fb11d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:42:21.621995 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-0dd784b4-bf52-4a9a-9841-f54aa47fb11d None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:42:21.623235 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-0dd784b4-bf52-4a9a-9841-f54aa47fb11d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:42:21.669963 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:42:21.719553 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 727/1483] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 10:42:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:42:21.756451 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36df62af-c28d-4ef8-bb47-633d2e721bd6 None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:42:21.757990 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-36df62af-c28d-4ef8-bb47-633d2e721bd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:42:21.781916 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:42:21.860320 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36df62af-c28d-4ef8-bb47-633d2e721bd6 None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:42:21.931111 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc3a583-caa5-4052-8ee5-73734a240ecf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7922d0a1a7c4e808d0b2a80dca0da40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:42:21.931725 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc3a583-caa5-4052-8ee5-73734a240ecf service neutron] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:42:21.932006 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc3a583-caa5-4052-8ee5-73734a240ecf service neutron] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:42:21.932279 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc3a583-caa5-4052-8ee5-73734a240ecf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:42:21.934492 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-3bc3a583-caa5-4052-8ee5-73734a240ecf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:42:22.009369 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:42:22.044726 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc3a583-caa5-4052-8ee5-73734a240ecf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:42:22.046179 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc3a583-caa5-4052-8ee5-73734a240ecf service neutron] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:42:22.064761 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 757/1484] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:42:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 10:42:24.111787 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242e0be2-7497-4a59-b6f6-c03cb1d0ab2c None None] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:42:24.112330 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242e0be2-7497-4a59-b6f6-c03cb1d0ab2c None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:42:24.112330 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242e0be2-7497-4a59-b6f6-c03cb1d0ab2c None None] PATH_INFO: `/` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:42:24.113680 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 728/1485] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 10:42:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 10:42:24.118926 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1db17f-f3fb-4306-a750-2a7fdd4446c5 None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:42:24.119350 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1db17f-f3fb-4306-a750-2a7fdd4446c5 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:42:24.119484 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1db17f-f3fb-4306-a750-2a7fdd4446c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:42:24.185500 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-8c1db17f-f3fb-4306-a750-2a7fdd4446c5 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:42:24.186725 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-8c1db17f-f3fb-4306-a750-2a7fdd4446c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:42:24.283058 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 758/1486] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 10:42:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:42:24.320267 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242e0be2-7497-4a59-b6f6-c03cb1d0ab2c None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:42:24.321780 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-242e0be2-7497-4a59-b6f6-c03cb1d0ab2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:42:24.427211 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242e0be2-7497-4a59-b6f6-c03cb1d0ab2c None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:42:24.497909 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:42:24.499712 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b19889-258c-44c8-9e98-a99359fb659f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7922d0a1a7c4e808d0b2a80dca0da40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:42:24.500315 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b19889-258c-44c8-9e98-a99359fb659f service neutron] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:42:24.500815 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b19889-258c-44c8-9e98-a99359fb659f service neutron] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:42:24.501271 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b19889-258c-44c8-9e98-a99359fb659f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:42:24.503151 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-98b19889-258c-44c8-9e98-a99359fb659f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:42:24.610841 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b19889-258c-44c8-9e98-a99359fb659f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:42:24.612271 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b19889-258c-44c8-9e98-a99359fb659f service neutron] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:42:24.631149 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 729/1487] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:42:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 10:42:26.772656 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7bb6d4-b34a-4a0d-bb50-78a81ac31148 None None] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:42:26.773307 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7bb6d4-b34a-4a0d-bb50-78a81ac31148 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:42:26.773307 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7bb6d4-b34a-4a0d-bb50-78a81ac31148 None None] PATH_INFO: `/` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:42:26.774660 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 759/1488] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 10:42:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 10:42:26.780682 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f59cc21-0e58-468e-b4b8-92947715b3ab None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:42:26.781132 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f59cc21-0e58-468e-b4b8-92947715b3ab None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:42:26.781273 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f59cc21-0e58-468e-b4b8-92947715b3ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:42:26.846753 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-0f59cc21-0e58-468e-b4b8-92947715b3ab None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:42:26.847964 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-0f59cc21-0e58-468e-b4b8-92947715b3ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:42:26.899591 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:42:26.943794 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 730/1489] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 10:42:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 10:42:26.984552 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7bb6d4-b34a-4a0d-bb50-78a81ac31148 None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:42:26.986121 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-6f7bb6d4-b34a-4a0d-bb50-78a81ac31148 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:42:27.022754 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:42:27.111844 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7bb6d4-b34a-4a0d-bb50-78a81ac31148 None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:42:27.184766 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7562ec61-bc6f-424a-9c45-3c4685f2e8fc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7922d0a1a7c4e808d0b2a80dca0da40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:42:27.185386 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7562ec61-bc6f-424a-9c45-3c4685f2e8fc service neutron] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:42:27.185673 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7562ec61-bc6f-424a-9c45-3c4685f2e8fc service neutron] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:42:27.185954 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7562ec61-bc6f-424a-9c45-3c4685f2e8fc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:42:27.187618 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-7562ec61-bc6f-424a-9c45-3c4685f2e8fc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:42:27.270675 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:42:27.300018 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7562ec61-bc6f-424a-9c45-3c4685f2e8fc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:42:27.301855 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7562ec61-bc6f-424a-9c45-3c4685f2e8fc service neutron] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:42:27.319955 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 760/1490] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:42:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 10:42:29.616866 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6057573-0d70-4d22-bd8d-ac3a422f97e5 None None] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:42:29.617390 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6057573-0d70-4d22-bd8d-ac3a422f97e5 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:42:29.617390 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6057573-0d70-4d22-bd8d-ac3a422f97e5 None None] PATH_INFO: `/` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:42:29.618731 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 731/1491] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 10:42:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 10:42:29.624741 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb16012-1f4c-4135-a8df-7c7a49a0ed42 None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:42:29.625154 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb16012-1f4c-4135-a8df-7c7a49a0ed42 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:42:29.625364 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb16012-1f4c-4135-a8df-7c7a49a0ed42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:42:29.691417 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-feb16012-1f4c-4135-a8df-7c7a49a0ed42 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:42:29.692672 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-feb16012-1f4c-4135-a8df-7c7a49a0ed42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:42:29.788698 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 761/1492] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 10:42:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:42:29.827639 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6057573-0d70-4d22-bd8d-ac3a422f97e5 None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:42:29.829223 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-a6057573-0d70-4d22-bd8d-ac3a422f97e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:42:29.931274 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6057573-0d70-4d22-bd8d-ac3a422f97e5 None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:42:30.004558 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e451f3-9c4b-4e02-9fc0-0e314700fec5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7922d0a1a7c4e808d0b2a80dca0da40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:42:30.005448 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e451f3-9c4b-4e02-9fc0-0e314700fec5 service neutron] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:42:30.005727 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e451f3-9c4b-4e02-9fc0-0e314700fec5 service neutron] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:42:30.005998 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e451f3-9c4b-4e02-9fc0-0e314700fec5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:42:30.007656 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-62e451f3-9c4b-4e02-9fc0-0e314700fec5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:42:30.018267 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:42:30.120305 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e451f3-9c4b-4e02-9fc0-0e314700fec5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:42:30.121779 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e451f3-9c4b-4e02-9fc0-0e314700fec5 service neutron] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:42:30.140382 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 732/1493] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:42:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 10:42:39.786967 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=82 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:42:40.138132 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=50 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:44:04.595356 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7945cb04-10c8-47b9-ac33-97a2c2ef50fe None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:44:04.595949 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7945cb04-10c8-47b9-ac33-97a2c2ef50fe None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:44:04.595949 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7945cb04-10c8-47b9-ac33-97a2c2ef50fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:44:04.670681 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-7945cb04-10c8-47b9-ac33-97a2c2ef50fe None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:44:04.671916 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-7945cb04-10c8-47b9-ac33-97a2c2ef50fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:44:04.767749 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 762/1494] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:44:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:44:04.777643 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312426ff-29c5-494b-99c7-19a7f306ec12 None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:44:04.778151 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312426ff-29c5-494b-99c7-19a7f306ec12 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:44:04.778263 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312426ff-29c5-494b-99c7-19a7f306ec12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:44:04.851320 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-312426ff-29c5-494b-99c7-19a7f306ec12 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:44:04.852441 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-312426ff-29c5-494b-99c7-19a7f306ec12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:44:04.946751 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 733/1495] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:44:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:44:04.951387 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7945cb04-10c8-47b9-ac33-97a2c2ef50fe None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:44:04.952626 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-7945cb04-10c8-47b9-ac33-97a2c2ef50fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:44:04.964110 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7945cb04-10c8-47b9-ac33-97a2c2ef50fe None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:44:05.024806 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:44:05.060238 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05afd5b-48ee-4d9f-9e16-7de8779965ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:44:05.060858 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05afd5b-48ee-4d9f-9e16-7de8779965ec admin admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:44:05.061140 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05afd5b-48ee-4d9f-9e16-7de8779965ec admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:44:05.061415 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05afd5b-48ee-4d9f-9e16-7de8779965ec admin admin] PATH_INFO: `/v3/domains` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:44:05.062103 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05afd5b-48ee-4d9f-9e16-7de8779965ec admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86827) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:44:05.062406 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05afd5b-48ee-4d9f-9e16-7de8779965ec admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:44:05.063453 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: WARNING py.warnings [None req-c05afd5b-48ee-4d9f-9e16-7de8779965ec admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 27 10:44:05.063453 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: warnings.warn(msg) Jul 27 10:44:05.063453 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: Jul 27 10:44:05.064329 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05afd5b-48ee-4d9f-9e16-7de8779965ec admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:44:05.075334 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 763/1496] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Jul 27 10:44:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:44:05.083716 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312426ff-29c5-494b-99c7-19a7f306ec12 None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:44:05.093375 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312426ff-29c5-494b-99c7-19a7f306ec12 None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:44:05.153958 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:44:05.186945 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cf6ead-41a7-4be8-885a-ece9fc122377 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:44:05.187567 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cf6ead-41a7-4be8-885a-ece9fc122377 admin admin] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:44:05.187847 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cf6ead-41a7-4be8-885a-ece9fc122377 admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:44:05.188121 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cf6ead-41a7-4be8-885a-ece9fc122377 admin admin] PATH_INFO: `/v3/projects` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:44:05.188790 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21cf6ead-41a7-4be8-885a-ece9fc122377 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:44:05.190517 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: WARNING py.warnings [None req-21cf6ead-41a7-4be8-885a-ece9fc122377 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 27 10:44:05.190517 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: warnings.warn(msg) Jul 27 10:44:05.190517 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: Jul 27 10:44:05.191734 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21cf6ead-41a7-4be8-885a-ece9fc122377 admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:44:05.246277 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 734/1497] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Jul 27 10:44:05 2022] POST /identity/v3/projects => generated 377 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 10:44:05.250527 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05afd5b-48ee-4d9f-9e16-7de8779965ec admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:44:05.252095 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-c05afd5b-48ee-4d9f-9e16-7de8779965ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:44:05.285100 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05afd5b-48ee-4d9f-9e16-7de8779965ec admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:44:05.376940 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d932804-7593-42eb-b80a-2ea7a857df8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:44:05.377557 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d932804-7593-42eb-b80a-2ea7a857df8a admin admin] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:44:05.377838 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d932804-7593-42eb-b80a-2ea7a857df8a admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:44:05.378111 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d932804-7593-42eb-b80a-2ea7a857df8a admin admin] PATH_INFO: `/v3/users` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:44:05.378805 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d932804-7593-42eb-b80a-2ea7a857df8a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:44:05.380081 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: WARNING py.warnings [None req-2d932804-7593-42eb-b80a-2ea7a857df8a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 27 10:44:05.380081 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: warnings.warn(msg) Jul 27 10:44:05.380081 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: Jul 27 10:44:05.381271 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d932804-7593-42eb-b80a-2ea7a857df8a admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:44:05.423113 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 764/1498] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Jul 27 10:44:05 2022] POST /identity/v3/users => generated 289 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 10:44:05.427105 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cf6ead-41a7-4be8-885a-ece9fc122377 admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:44:05.428614 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-21cf6ead-41a7-4be8-885a-ece9fc122377 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:44:05.491354 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cf6ead-41a7-4be8-885a-ece9fc122377 admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:44:05.572340 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:44:05.583969 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81347c45-083f-4d56-95bc-63444b4a4e32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:44:05.584583 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81347c45-083f-4d56-95bc-63444b4a4e32 admin admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:44:05.584866 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81347c45-083f-4d56-95bc-63444b4a4e32 admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:44:05.585133 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81347c45-083f-4d56-95bc-63444b4a4e32 admin admin] PATH_INFO: `/v3/roles` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:44:05.585821 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81347c45-083f-4d56-95bc-63444b4a4e32 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:44:05.586867 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: WARNING py.warnings [None req-81347c45-083f-4d56-95bc-63444b4a4e32 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 27 10:44:05.586867 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: warnings.warn(msg) Jul 27 10:44:05.586867 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: Jul 27 10:44:05.587745 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81347c45-083f-4d56-95bc-63444b4a4e32 admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:44:05.600486 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 735/1499] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 10:44:05 2022] GET /identity/v3/roles => generated 1420 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 10:44:05.604561 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d932804-7593-42eb-b80a-2ea7a857df8a admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:44:05.606085 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-2d932804-7593-42eb-b80a-2ea7a857df8a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:44:05.617541 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d932804-7593-42eb-b80a-2ea7a857df8a admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:44:05.643342 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:44:05.716501 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb672bf-d750-4ef0-8124-ae59bfe1faf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:44:05.717129 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb672bf-d750-4ef0-8124-ae59bfe1faf2 admin admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:44:05.717411 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb672bf-d750-4ef0-8124-ae59bfe1faf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:44:05.717687 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb672bf-d750-4ef0-8124-ae59bfe1faf2 admin admin] PATH_INFO: `/v3/roles` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:44:05.718383 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb672bf-d750-4ef0-8124-ae59bfe1faf2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:44:05.719512 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: WARNING py.warnings [None req-8eb672bf-d750-4ef0-8124-ae59bfe1faf2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 27 10:44:05.719512 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: warnings.warn(msg) Jul 27 10:44:05.719512 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: Jul 27 10:44:05.720400 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb672bf-d750-4ef0-8124-ae59bfe1faf2 admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:44:05.732831 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 765/1500] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 10:44:05 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 10:44:05.737057 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81347c45-083f-4d56-95bc-63444b4a4e32 admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:44:05.738567 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-81347c45-083f-4d56-95bc-63444b4a4e32 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:44:05.771363 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81347c45-083f-4d56-95bc-63444b4a4e32 admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:44:05.873389 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1df730-5fac-435d-bb65-253ecd1d7b4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:44:05.874084 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1df730-5fac-435d-bb65-253ecd1d7b4c admin admin] REQUEST_METHOD: `PUT` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:44:05.874332 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1df730-5fac-435d-bb65-253ecd1d7b4c admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:44:05.874621 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1df730-5fac-435d-bb65-253ecd1d7b4c admin admin] PATH_INFO: `/v3/projects/614b1034255d445e9647fa79493753c8/users/ce344eaa107443908d7af552cb3e2bc2/roles/4c9fc0b3baed441da2126be41cc6b04a` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:44:05.896853 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1df730-5fac-435d-bb65-253ecd1d7b4c admin admin] RBAC: Authorizing `identity:create_grant(project_id=614b1034255d445e9647fa79493753c8, user_id=ce344eaa107443908d7af552cb3e2bc2, role_id=4c9fc0b3baed441da2126be41cc6b04a)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:44:05.897874 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: WARNING py.warnings [None req-ca1df730-5fac-435d-bb65-253ecd1d7b4c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 27 10:44:05.897874 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: warnings.warn(msg) Jul 27 10:44:05.897874 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: Jul 27 10:44:05.899863 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1df730-5fac-435d-bb65-253ecd1d7b4c admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:44:05.923785 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 736/1501] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 10:44:05 2022] PUT /identity/v3/projects/614b1034255d445e9647fa79493753c8/users/ce344eaa107443908d7af552cb3e2bc2/roles/4c9fc0b3baed441da2126be41cc6b04a => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:44:05.936332 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc7ddeb-9cf6-4974-be88-cd868f800075 None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:44:05.936743 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc7ddeb-9cf6-4974-be88-cd868f800075 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:44:05.937036 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc7ddeb-9cf6-4974-be88-cd868f800075 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:44:05.994518 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-1bc7ddeb-9cf6-4974-be88-cd868f800075 None None] MFA Rules not processed for user `ce344eaa107443908d7af552cb3e2bc2`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:44:05.996187 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-1bc7ddeb-9cf6-4974-be88-cd868f800075 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:44:06.072062 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 766/1502] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:44:05 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:44:06.085973 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5efbd69-3024-43e9-aefc-a406833751e1 None None] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:44:06.086279 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5efbd69-3024-43e9-aefc-a406833751e1 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:44:06.086828 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5efbd69-3024-43e9-aefc-a406833751e1 None None] PATH_INFO: `/` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:44:06.088917 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 737/1503] 10.222.0.36 () {54 vars in 964 bytes} [Wed Jul 27 10:44:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 10:44:06.094219 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52c4d98-9486-45cf-a256-e321c5ff6677 None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:44:06.094514 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52c4d98-9486-45cf-a256-e321c5ff6677 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:44:06.094832 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52c4d98-9486-45cf-a256-e321c5ff6677 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:44:06.125899 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:44:06.171814 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-d52c4d98-9486-45cf-a256-e321c5ff6677 None None] MFA Rules not processed for user `864d35f268cf45148756109759bb2743`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:44:06.173019 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-d52c4d98-9486-45cf-a256-e321c5ff6677 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:44:06.247461 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 767/1504] 10.222.0.36 () {58 vars in 1061 bytes} [Wed Jul 27 10:44:06 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:44:06.255551 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5efbd69-3024-43e9-aefc-a406833751e1 None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:44:06.257077 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-a5efbd69-3024-43e9-aefc-a406833751e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:44:06.306019 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:44:06.360923 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5efbd69-3024-43e9-aefc-a406833751e1 None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:44:06.442199 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d997f72-540b-4f6e-9afc-b095d349e188 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '864d35f268cf45148756109759bb2743', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:44:06.442859 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d997f72-540b-4f6e-9afc-b095d349e188 service cinder] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:44:06.443150 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d997f72-540b-4f6e-9afc-b095d349e188 service cinder] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:44:06.443432 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d997f72-540b-4f6e-9afc-b095d349e188 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:44:06.445068 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-7d997f72-540b-4f6e-9afc-b095d349e188 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:44:06.551191 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d997f72-540b-4f6e-9afc-b095d349e188 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:44:06.552654 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d997f72-540b-4f6e-9afc-b095d349e188 service cinder] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:44:06.556095 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:44:06.571094 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 738/1505] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:44:06 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 10:44:06.617676 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e63d781-8077-488f-9a34-2cae16ebd0f0 None None] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:44:06.618111 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e63d781-8077-488f-9a34-2cae16ebd0f0 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:44:06.618289 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e63d781-8077-488f-9a34-2cae16ebd0f0 None None] PATH_INFO: `/` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:44:06.619594 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 768/1506] 10.222.0.36 () {52 vars in 830 bytes} [Wed Jul 27 10:44:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 10:44:06.623630 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d997f72-540b-4f6e-9afc-b095d349e188 service cinder] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:44:06.624875 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-7d997f72-540b-4f6e-9afc-b095d349e188 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:44:06.724617 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d997f72-540b-4f6e-9afc-b095d349e188 service cinder] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:44:06.817467 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7858325b-af54-4c86-9db2-47425e694fb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:44:06.818063 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7858325b-af54-4c86-9db2-47425e694fb2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:44:06.818346 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7858325b-af54-4c86-9db2-47425e694fb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:44:06.818711 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7858325b-af54-4c86-9db2-47425e694fb2 admin admin] PATH_INFO: `/v3/users/ce344eaa107443908d7af552cb3e2bc2` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:44:06.830997 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7858325b-af54-4c86-9db2-47425e694fb2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ce344eaa107443908d7af552cb3e2bc2)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:44:06.832021 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: WARNING py.warnings [None req-7858325b-af54-4c86-9db2-47425e694fb2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 27 10:44:06.832021 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: warnings.warn(msg) Jul 27 10:44:06.832021 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: Jul 27 10:44:06.833230 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7858325b-af54-4c86-9db2-47425e694fb2 admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:44:06.843182 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:44:06.895409 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-7858325b-af54-4c86-9db2-47425e694fb2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ce344eaa107443908d7af552cb3e2bc2'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:44:06.908155 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-7858325b-af54-4c86-9db2-47425e694fb2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ce344eaa107443908d7af552cb3e2bc2'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:44:06.916458 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-7858325b-af54-4c86-9db2-47425e694fb2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ce344eaa107443908d7af552cb3e2bc2'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:44:06.917428 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-7858325b-af54-4c86-9db2-47425e694fb2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ce344eaa107443908d7af552cb3e2bc2'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:44:06.917934 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-7858325b-af54-4c86-9db2-47425e694fb2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ce344eaa107443908d7af552cb3e2bc2'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:44:06.941800 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 739/1507] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Jul 27 10:44:06 2022] DELETE /identity/v3/users/ce344eaa107443908d7af552cb3e2bc2 => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:44:06.946439 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbc8bf6-7dd1-4a9b-8c8f-1933e36242d6 None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:44:06.946758 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbc8bf6-7dd1-4a9b-8c8f-1933e36242d6 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:44:06.947028 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbc8bf6-7dd1-4a9b-8c8f-1933e36242d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:44:07.004423 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-5dbc8bf6-7dd1-4a9b-8c8f-1933e36242d6 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:44:07.005634 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-5dbc8bf6-7dd1-4a9b-8c8f-1933e36242d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:44:07.080366 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:44:07.094772 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 769/1508] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:44:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:44:07.104149 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7858325b-af54-4c86-9db2-47425e694fb2 admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:44:07.105479 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-7858325b-af54-4c86-9db2-47425e694fb2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:44:07.206718 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7858325b-af54-4c86-9db2-47425e694fb2 admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:44:07.259824 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:44:07.281051 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586ca02b-52d1-4a6f-8c81-7eae85a1b9b1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7922d0a1a7c4e808d0b2a80dca0da40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:44:07.281671 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586ca02b-52d1-4a6f-8c81-7eae85a1b9b1 service neutron] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:44:07.281950 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586ca02b-52d1-4a6f-8c81-7eae85a1b9b1 service neutron] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:44:07.282219 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586ca02b-52d1-4a6f-8c81-7eae85a1b9b1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:44:07.283863 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-586ca02b-52d1-4a6f-8c81-7eae85a1b9b1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:44:07.404895 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586ca02b-52d1-4a6f-8c81-7eae85a1b9b1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:44:07.406849 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586ca02b-52d1-4a6f-8c81-7eae85a1b9b1 service neutron] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:44:07.429832 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 740/1509] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:44:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 10:44:07.760163 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbc8bf6-7dd1-4a9b-8c8f-1933e36242d6 None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:44:07.761946 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-5dbc8bf6-7dd1-4a9b-8c8f-1933e36242d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:44:07.774930 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbc8bf6-7dd1-4a9b-8c8f-1933e36242d6 None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:44:07.871200 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc051f48-4b46-40ac-9c18-bdf341119307 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:44:07.871851 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc051f48-4b46-40ac-9c18-bdf341119307 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:44:07.872141 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc051f48-4b46-40ac-9c18-bdf341119307 admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:44:07.872417 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc051f48-4b46-40ac-9c18-bdf341119307 admin admin] PATH_INFO: `/v3/projects/614b1034255d445e9647fa79493753c8` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:44:07.882368 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc051f48-4b46-40ac-9c18-bdf341119307 admin admin] RBAC: Authorizing `identity:delete_project(project_id=614b1034255d445e9647fa79493753c8)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:44:07.883428 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: WARNING py.warnings [None req-dc051f48-4b46-40ac-9c18-bdf341119307 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 27 10:44:07.883428 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: warnings.warn(msg) Jul 27 10:44:07.883428 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: Jul 27 10:44:07.884627 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc051f48-4b46-40ac-9c18-bdf341119307 admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:44:07.943129 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-dc051f48-4b46-40ac-9c18-bdf341119307 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '614b1034255d445e9647fa79493753c8', 'request_id': 'req-dc051f48-4b46-40ac-9c18-bdf341119307'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:44:07.956181 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-dc051f48-4b46-40ac-9c18-bdf341119307 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '614b1034255d445e9647fa79493753c8', 'request_id': 'req-dc051f48-4b46-40ac-9c18-bdf341119307'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:44:07.966522 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-dc051f48-4b46-40ac-9c18-bdf341119307 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '614b1034255d445e9647fa79493753c8', 'request_id': 'req-dc051f48-4b46-40ac-9c18-bdf341119307'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:44:07.990823 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-dc051f48-4b46-40ac-9c18-bdf341119307 admin admin] The token cache is being invalidate because project 614b1034255d445e9647fa79493753c8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86827) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 10:44:07.991113 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-dc051f48-4b46-40ac-9c18-bdf341119307 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:44:07.991632 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-dc051f48-4b46-40ac-9c18-bdf341119307 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:44:07.992952 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 770/1510] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Jul 27 10:44:07 2022] DELETE /identity/v3/projects/614b1034255d445e9647fa79493753c8 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:44:17.427573 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=62 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:44:17.988809 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:45:50.715596 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8770f399-bed0-448a-95fe-08194bd08a3c None None] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:45:50.716194 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8770f399-bed0-448a-95fe-08194bd08a3c None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:45:50.716305 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8770f399-bed0-448a-95fe-08194bd08a3c None None] PATH_INFO: `/` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:45:50.717654 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 741/1511] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 27 10:45:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 10:45:50.722564 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4600ec77-bb30-4dfc-8043-f5d03a738795 None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:45:50.723025 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4600ec77-bb30-4dfc-8043-f5d03a738795 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:45:50.723140 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4600ec77-bb30-4dfc-8043-f5d03a738795 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:45:50.808099 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-4600ec77-bb30-4dfc-8043-f5d03a738795 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:45:50.809309 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-4600ec77-bb30-4dfc-8043-f5d03a738795 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:45:50.914145 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 771/1512] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 27 10:45:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:45:51.068004 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8770f399-bed0-448a-95fe-08194bd08a3c None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:45:51.070095 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-8770f399-bed0-448a-95fe-08194bd08a3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:45:51.200150 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8770f399-bed0-448a-95fe-08194bd08a3c None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:45:51.206642 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c06145-caf2-48f6-96a4-1611e449ee8c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e059d8b6c28c4fcba925d05d7c98e4ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:45:51.207280 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c06145-caf2-48f6-96a4-1611e449ee8c service nova] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:45:51.207559 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c06145-caf2-48f6-96a4-1611e449ee8c service nova] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:45:51.207837 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c06145-caf2-48f6-96a4-1611e449ee8c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:45:51.215903 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c06145-caf2-48f6-96a4-1611e449ee8c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:45:51.218006 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c06145-caf2-48f6-96a4-1611e449ee8c service nova] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:45:51.227208 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 742/1513] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:45:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 10:45:52.061716 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a701c586-ad7b-4ac3-8f63-d62acafc1dac None None] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:45:52.062256 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a701c586-ad7b-4ac3-8f63-d62acafc1dac None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:45:52.062357 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a701c586-ad7b-4ac3-8f63-d62acafc1dac None None] PATH_INFO: `/` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:45:52.063888 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 772/1514] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 27 10:45:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 10:45:52.069076 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d262a3-5a93-4915-a1ba-c21fb0bdef39 None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:45:52.069604 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d262a3-5a93-4915-a1ba-c21fb0bdef39 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:45:52.069604 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d262a3-5a93-4915-a1ba-c21fb0bdef39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:45:52.090248 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-39d262a3-5a93-4915-a1ba-c21fb0bdef39 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:45:52.091474 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-39d262a3-5a93-4915-a1ba-c21fb0bdef39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:45:52.099666 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 743/1515] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 27 10:45:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:45:52.252378 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a701c586-ad7b-4ac3-8f63-d62acafc1dac None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:45:52.267595 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a701c586-ad7b-4ac3-8f63-d62acafc1dac None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:45:52.273072 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60c6b46-76e2-4780-870f-5cd77823fb2e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e059d8b6c28c4fcba925d05d7c98e4ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:45:52.273717 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60c6b46-76e2-4780-870f-5cd77823fb2e service nova] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:45:52.274002 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60c6b46-76e2-4780-870f-5cd77823fb2e service nova] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:45:52.274292 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60c6b46-76e2-4780-870f-5cd77823fb2e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:45:52.282633 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60c6b46-76e2-4780-870f-5cd77823fb2e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:45:52.284290 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60c6b46-76e2-4780-870f-5cd77823fb2e service nova] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:45:52.296053 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 773/1516] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:45:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 10:45:52.687772 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41632fbb-8191-46ba-a789-ce0d105bbcd8 None None] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:45:52.687987 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41632fbb-8191-46ba-a789-ce0d105bbcd8 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:45:52.688270 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41632fbb-8191-46ba-a789-ce0d105bbcd8 None None] PATH_INFO: `/v3` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:45:52.689600 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 744/1517] 10.222.0.69 () {54 vars in 925 bytes} [Wed Jul 27 10:45:52 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:45:52.695775 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6045674f-8179-484d-9080-2f9393697f50 None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:45:52.696070 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6045674f-8179-484d-9080-2f9393697f50 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:45:52.696355 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6045674f-8179-484d-9080-2f9393697f50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:45:52.755347 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-6045674f-8179-484d-9080-2f9393697f50 None None] MFA Rules not processed for user `16cc6179e9a04b6398fbacb40ad1c555`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:45:52.756552 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-6045674f-8179-484d-9080-2f9393697f50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:45:52.774425 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:45:52.829114 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 774/1518] 10.222.0.69 () {58 vars in 1015 bytes} [Wed Jul 27 10:45:52 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 10:45:56.615924 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeea0975-7467-4422-8719-b22e476d8e6d None None] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:45:56.616487 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeea0975-7467-4422-8719-b22e476d8e6d None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:45:56.616606 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeea0975-7467-4422-8719-b22e476d8e6d None None] PATH_INFO: `/` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:45:56.618168 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 745/1519] 10.222.0.36 () {54 vars in 961 bytes} [Wed Jul 27 10:45:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 10:45:56.623113 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d6b089-9906-455e-8ddc-1e6902139e0d None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:45:56.623693 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d6b089-9906-455e-8ddc-1e6902139e0d None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:45:56.623693 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d6b089-9906-455e-8ddc-1e6902139e0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:45:56.643954 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-16d6b089-9906-455e-8ddc-1e6902139e0d None None] MFA Rules not processed for user `16cc6179e9a04b6398fbacb40ad1c555`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:45:56.645244 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-16d6b089-9906-455e-8ddc-1e6902139e0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:45:56.653810 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 775/1520] 10.222.0.36 () {58 vars in 1058 bytes} [Wed Jul 27 10:45:56 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:45:56.662008 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeea0975-7467-4422-8719-b22e476d8e6d None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:45:56.675332 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeea0975-7467-4422-8719-b22e476d8e6d None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:45:56.680280 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ca0df9-d586-4dfb-b362-4bb99f5ddb45 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16cc6179e9a04b6398fbacb40ad1c555', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:45:56.680891 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ca0df9-d586-4dfb-b362-4bb99f5ddb45 service placement] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:45:56.681177 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ca0df9-d586-4dfb-b362-4bb99f5ddb45 service placement] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:45:56.681456 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ca0df9-d586-4dfb-b362-4bb99f5ddb45 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:45:56.689841 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ca0df9-d586-4dfb-b362-4bb99f5ddb45 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:45:56.691444 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ca0df9-d586-4dfb-b362-4bb99f5ddb45 service placement] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:45:56.701020 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 746/1521] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:45:56 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 10:46:03.495074 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa71ddc-9ca2-40d9-bb41-09f0d2bc66d5 None None] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:46:03.495643 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa71ddc-9ca2-40d9-bb41-09f0d2bc66d5 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:46:03.495643 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa71ddc-9ca2-40d9-bb41-09f0d2bc66d5 None None] PATH_INFO: `/` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:46:03.497039 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 776/1522] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 27 10:46:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 10:46:03.502077 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03df0f2-c7ab-455c-aa05-81646680fcd5 None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:46:03.502539 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03df0f2-c7ab-455c-aa05-81646680fcd5 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:46:03.502671 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03df0f2-c7ab-455c-aa05-81646680fcd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:46:03.522305 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-f03df0f2-c7ab-455c-aa05-81646680fcd5 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:46:03.523517 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-f03df0f2-c7ab-455c-aa05-81646680fcd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:46:03.531877 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 747/1523] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 27 10:46:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:46:03.687787 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa71ddc-9ca2-40d9-bb41-09f0d2bc66d5 None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:46:03.702282 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa71ddc-9ca2-40d9-bb41-09f0d2bc66d5 None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:46:03.708307 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4309005f-a45a-43d5-8da4-32b870f09cd3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e059d8b6c28c4fcba925d05d7c98e4ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:46:03.708919 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4309005f-a45a-43d5-8da4-32b870f09cd3 service nova] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:46:03.709203 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4309005f-a45a-43d5-8da4-32b870f09cd3 service nova] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:46:03.709477 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4309005f-a45a-43d5-8da4-32b870f09cd3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:46:03.717508 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4309005f-a45a-43d5-8da4-32b870f09cd3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:46:03.719838 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4309005f-a45a-43d5-8da4-32b870f09cd3 service nova] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:46:03.729231 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 777/1524] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:46:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 10:46:04.551217 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329161f6-e311-427c-a54e-31d75143e124 None None] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:46:04.551814 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329161f6-e311-427c-a54e-31d75143e124 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:46:04.551814 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329161f6-e311-427c-a54e-31d75143e124 None None] PATH_INFO: `/` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:46:04.553102 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 748/1525] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 27 10:46:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 10:46:04.558014 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6012e6-de5e-491f-adbc-3ff5dcdbeae9 None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:46:04.558461 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6012e6-de5e-491f-adbc-3ff5dcdbeae9 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:46:04.558578 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6012e6-de5e-491f-adbc-3ff5dcdbeae9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:46:04.578680 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-0a6012e6-de5e-491f-adbc-3ff5dcdbeae9 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:46:04.579886 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-0a6012e6-de5e-491f-adbc-3ff5dcdbeae9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:46:04.587748 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 778/1526] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 27 10:46:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:46:04.737775 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329161f6-e311-427c-a54e-31d75143e124 None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:46:04.752580 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329161f6-e311-427c-a54e-31d75143e124 None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:46:04.758278 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0cab97-5885-4525-b705-16032208ac08 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e059d8b6c28c4fcba925d05d7c98e4ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:46:04.758940 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0cab97-5885-4525-b705-16032208ac08 service nova] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:46:04.759229 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0cab97-5885-4525-b705-16032208ac08 service nova] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:46:04.759506 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0cab97-5885-4525-b705-16032208ac08 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:46:04.767995 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0cab97-5885-4525-b705-16032208ac08 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:46:04.769569 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0cab97-5885-4525-b705-16032208ac08 service nova] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:46:04.778896 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 749/1527] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:46:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 10:46:08.747522 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40bc9e7-7bea-4631-95df-6170abd68ca6 None None] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:46:08.748062 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40bc9e7-7bea-4631-95df-6170abd68ca6 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:46:08.748062 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40bc9e7-7bea-4631-95df-6170abd68ca6 None None] PATH_INFO: `/` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:46:08.750625 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 779/1528] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 27 10:46:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 10:46:08.755595 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b30fffa-b6bc-4384-8d78-9bc56afb93fc None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:46:08.756027 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b30fffa-b6bc-4384-8d78-9bc56afb93fc None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:46:08.756160 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b30fffa-b6bc-4384-8d78-9bc56afb93fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:46:08.775297 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-6b30fffa-b6bc-4384-8d78-9bc56afb93fc None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:46:08.776493 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-6b30fffa-b6bc-4384-8d78-9bc56afb93fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:46:08.784397 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 750/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 27 10:46:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:46:08.856731 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40bc9e7-7bea-4631-95df-6170abd68ca6 None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:46:08.858441 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-f40bc9e7-7bea-4631-95df-6170abd68ca6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:46:08.938847 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40bc9e7-7bea-4631-95df-6170abd68ca6 None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:46:08.942375 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbed5fa-98bd-4a17-8676-1c2aca0c5a3c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7922d0a1a7c4e808d0b2a80dca0da40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:46:08.943018 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbed5fa-98bd-4a17-8676-1c2aca0c5a3c service neutron] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:46:08.943297 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbed5fa-98bd-4a17-8676-1c2aca0c5a3c service neutron] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:46:08.943566 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbed5fa-98bd-4a17-8676-1c2aca0c5a3c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:46:08.953418 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbed5fa-98bd-4a17-8676-1c2aca0c5a3c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:46:08.954864 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbed5fa-98bd-4a17-8676-1c2aca0c5a3c service neutron] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:46:08.964124 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 780/1530] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:46:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 10:46:11.010206 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac221ad6-3abc-4e57-a694-0f71f414eea0 None None] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:46:11.010784 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac221ad6-3abc-4e57-a694-0f71f414eea0 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:46:11.010891 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac221ad6-3abc-4e57-a694-0f71f414eea0 None None] PATH_INFO: `/` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:46:11.012189 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 751/1531] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 27 10:46:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 10:46:11.017315 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8c0085-99c1-4fa2-9964-5938b5861a6d None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:46:11.017753 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8c0085-99c1-4fa2-9964-5938b5861a6d None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:46:11.017868 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8c0085-99c1-4fa2-9964-5938b5861a6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:46:11.036685 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-af8c0085-99c1-4fa2-9964-5938b5861a6d None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:46:11.037876 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-af8c0085-99c1-4fa2-9964-5938b5861a6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:46:11.045806 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 781/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 27 10:46:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:46:11.082045 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac221ad6-3abc-4e57-a694-0f71f414eea0 None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:46:11.092816 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac221ad6-3abc-4e57-a694-0f71f414eea0 None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:46:11.097088 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f358ec6-53af-463e-b787-80179b7f3a40 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7922d0a1a7c4e808d0b2a80dca0da40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:46:11.097715 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f358ec6-53af-463e-b787-80179b7f3a40 service neutron] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:46:11.098001 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f358ec6-53af-463e-b787-80179b7f3a40 service neutron] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:46:11.098282 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f358ec6-53af-463e-b787-80179b7f3a40 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:46:11.107270 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f358ec6-53af-463e-b787-80179b7f3a40 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:46:11.108692 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f358ec6-53af-463e-b787-80179b7f3a40 service neutron] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:46:11.117930 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 752/1533] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:46:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 10:46:14.810935 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1abb0b-a1e0-49aa-abf9-eb3623c539e7 None None] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:46:14.811535 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1abb0b-a1e0-49aa-abf9-eb3623c539e7 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:46:14.811646 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1abb0b-a1e0-49aa-abf9-eb3623c539e7 None None] PATH_INFO: `/` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:46:14.813767 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 782/1534] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 27 10:46:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 10:46:14.819336 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a61ae0-3432-47da-b795-9d935c94354a None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:46:14.819758 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a61ae0-3432-47da-b795-9d935c94354a None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:46:14.819871 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a61ae0-3432-47da-b795-9d935c94354a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:46:14.838695 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-65a61ae0-3432-47da-b795-9d935c94354a None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:46:14.839876 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-65a61ae0-3432-47da-b795-9d935c94354a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:46:14.847650 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 753/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 27 10:46:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:46:15.149215 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1abb0b-a1e0-49aa-abf9-eb3623c539e7 None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:46:15.150917 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-2e1abb0b-a1e0-49aa-abf9-eb3623c539e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:46:15.215097 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:46:15.229081 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1abb0b-a1e0-49aa-abf9-eb3623c539e7 None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:46:15.232664 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b258f6-3d1b-48bd-9e4a-5b3588c19bd9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '864d35f268cf45148756109759bb2743', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:46:15.233280 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b258f6-3d1b-48bd-9e4a-5b3588c19bd9 service cinder] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:46:15.233562 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b258f6-3d1b-48bd-9e4a-5b3588c19bd9 service cinder] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:46:15.234441 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b258f6-3d1b-48bd-9e4a-5b3588c19bd9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:46:15.243369 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b258f6-3d1b-48bd-9e4a-5b3588c19bd9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:46:15.244786 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b258f6-3d1b-48bd-9e4a-5b3588c19bd9 service cinder] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:46:15.254405 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 783/1536] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:46:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 10:46:17.265698 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2197cb9-9678-444b-b4ce-0cce543e473a None None] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:46:17.266216 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2197cb9-9678-444b-b4ce-0cce543e473a None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:46:17.266216 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2197cb9-9678-444b-b4ce-0cce543e473a None None] PATH_INFO: `/` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:46:17.267571 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 754/1537] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 27 10:46:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 10:46:17.272566 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d348367-922e-4441-be4c-958b305b0478 None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:46:17.272999 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d348367-922e-4441-be4c-958b305b0478 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:46:17.273120 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d348367-922e-4441-be4c-958b305b0478 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:46:17.292665 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-0d348367-922e-4441-be4c-958b305b0478 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:46:17.293842 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-0d348367-922e-4441-be4c-958b305b0478 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:46:17.302356 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 784/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 27 10:46:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:46:17.604181 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2197cb9-9678-444b-b4ce-0cce543e473a None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:46:17.605845 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-f2197cb9-9678-444b-b4ce-0cce543e473a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:46:17.614205 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=84 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:46:17.620732 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2197cb9-9678-444b-b4ce-0cce543e473a None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:46:17.624840 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bbc41d-59a7-4b93-97b6-7517a294d7f4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '864d35f268cf45148756109759bb2743', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:46:17.625466 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bbc41d-59a7-4b93-97b6-7517a294d7f4 service cinder] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:46:17.625754 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bbc41d-59a7-4b93-97b6-7517a294d7f4 service cinder] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:46:17.626035 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bbc41d-59a7-4b93-97b6-7517a294d7f4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:46:17.635211 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82bbc41d-59a7-4b93-97b6-7517a294d7f4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:46:17.636671 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82bbc41d-59a7-4b93-97b6-7517a294d7f4 service cinder] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:46:17.646696 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 755/1539] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:46:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 10:46:27.288008 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=11 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:46:27.642448 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=5 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:47:47.202302 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4052e2-b6c5-49de-8a63-71cb0fae74a1 None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:47:47.203822 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4052e2-b6c5-49de-8a63-71cb0fae74a1 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:47:47.203822 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4052e2-b6c5-49de-8a63-71cb0fae74a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:47:47.228815 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-0c4052e2-b6c5-49de-8a63-71cb0fae74a1 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:47:47.230042 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-0c4052e2-b6c5-49de-8a63-71cb0fae74a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:47:47.243878 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 785/1540] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:47:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:47:47.254323 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e1456c-1f21-49b3-a8d7-ea0ed23c255f None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:47:47.255439 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e1456c-1f21-49b3-a8d7-ea0ed23c255f None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:47:47.255926 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e1456c-1f21-49b3-a8d7-ea0ed23c255f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:47:47.288128 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-c0e1456c-1f21-49b3-a8d7-ea0ed23c255f None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:47:47.289616 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-c0e1456c-1f21-49b3-a8d7-ea0ed23c255f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:47:47.302676 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 756/1541] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:47:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:47:47.307527 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4052e2-b6c5-49de-8a63-71cb0fae74a1 None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:47:47.322066 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4052e2-b6c5-49de-8a63-71cb0fae74a1 None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:47:47.330899 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d95aa7f-d040-4eec-a127-7aa0d3e8ccd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:47:47.331559 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d95aa7f-d040-4eec-a127-7aa0d3e8ccd8 admin admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:47:47.331843 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d95aa7f-d040-4eec-a127-7aa0d3e8ccd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:47:47.332121 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d95aa7f-d040-4eec-a127-7aa0d3e8ccd8 admin admin] PATH_INFO: `/v3/domains` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:47:47.332859 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d95aa7f-d040-4eec-a127-7aa0d3e8ccd8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86827) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:47:47.333169 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d95aa7f-d040-4eec-a127-7aa0d3e8ccd8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:47:47.334899 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d95aa7f-d040-4eec-a127-7aa0d3e8ccd8 admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:47:47.352181 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 786/1542] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Jul 27 10:47:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:47:47.360783 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e1456c-1f21-49b3-a8d7-ea0ed23c255f None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:47:47.370513 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e1456c-1f21-49b3-a8d7-ea0ed23c255f None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:47:47.382267 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c252154-f2ec-42cc-8d46-275fd968569e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:47:47.382936 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c252154-f2ec-42cc-8d46-275fd968569e admin admin] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:47:47.383227 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c252154-f2ec-42cc-8d46-275fd968569e admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:47:47.383494 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c252154-f2ec-42cc-8d46-275fd968569e admin admin] PATH_INFO: `/v3/projects` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:47:47.384175 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c252154-f2ec-42cc-8d46-275fd968569e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:47:47.386085 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c252154-f2ec-42cc-8d46-275fd968569e admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:47:47.401383 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf4dd6d-afca-43ca-911d-e92e3a7d4b0b None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:47:47.401705 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf4dd6d-afca-43ca-911d-e92e3a7d4b0b None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:47:47.401957 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf4dd6d-afca-43ca-911d-e92e3a7d4b0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:47:47.432489 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-acf4dd6d-afca-43ca-911d-e92e3a7d4b0b None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:47:47.433758 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-acf4dd6d-afca-43ca-911d-e92e3a7d4b0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:47:47.450639 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 787/1543] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:47:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 10:47:47.452566 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 757/1544] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Jul 27 10:47:47 2022] POST /identity/v3/projects => generated 355 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 10:47:47.458245 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf4dd6d-afca-43ca-911d-e92e3a7d4b0b None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:47:47.464182 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf143257-f249-4004-ad1b-3f6e9807fc79 None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:47:47.464478 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf143257-f249-4004-ad1b-3f6e9807fc79 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:47:47.464762 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf143257-f249-4004-ad1b-3f6e9807fc79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:47:47.474151 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf4dd6d-afca-43ca-911d-e92e3a7d4b0b None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:47:47.503429 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-cf143257-f249-4004-ad1b-3f6e9807fc79 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:47:47.504889 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-cf143257-f249-4004-ad1b-3f6e9807fc79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:47:47.597990 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097fec2b-cf93-4b72-bef5-430dbd67c959 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:47:47.598721 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097fec2b-cf93-4b72-bef5-430dbd67c959 admin admin] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:47:47.599009 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097fec2b-cf93-4b72-bef5-430dbd67c959 admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:47:47.599285 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097fec2b-cf93-4b72-bef5-430dbd67c959 admin admin] PATH_INFO: `/v3/users` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:47:47.599974 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097fec2b-cf93-4b72-bef5-430dbd67c959 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:47:47.601563 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 758/1545] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:47:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 10:47:47.604924 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097fec2b-cf93-4b72-bef5-430dbd67c959 admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:47:47.606141 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf143257-f249-4004-ad1b-3f6e9807fc79 None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:47:47.616299 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf143257-f249-4004-ad1b-3f6e9807fc79 None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:47:47.633345 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4756ea9b-f4cb-4126-8897-a05ee727787d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:47:47.634407 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4756ea9b-f4cb-4126-8897-a05ee727787d admin admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:47:47.634907 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4756ea9b-f4cb-4126-8897-a05ee727787d admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:47:47.635383 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4756ea9b-f4cb-4126-8897-a05ee727787d admin admin] PATH_INFO: `/v3/domains` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:47:47.636410 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4756ea9b-f4cb-4126-8897-a05ee727787d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86828) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:47:47.636925 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4756ea9b-f4cb-4126-8897-a05ee727787d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:47:47.638184 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: WARNING py.warnings [None req-4756ea9b-f4cb-4126-8897-a05ee727787d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 27 10:47:47.638184 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: warnings.warn(msg) Jul 27 10:47:47.638184 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: Jul 27 10:47:47.639460 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4756ea9b-f4cb-4126-8897-a05ee727787d admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:47:47.642988 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 788/1546] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Jul 27 10:47:47 2022] POST /identity/v3/users => generated 278 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 10:47:47.650621 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097fec2b-cf93-4b72-bef5-430dbd67c959 admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:47:47.659176 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 759/1547] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Jul 27 10:47:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:47:47.660800 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72930c95-6000-44f8-9512-0abe5189d4d3 None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:47:47.661291 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72930c95-6000-44f8-9512-0abe5189d4d3 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:47:47.661816 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72930c95-6000-44f8-9512-0abe5189d4d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:47:47.674550 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097fec2b-cf93-4b72-bef5-430dbd67c959 admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:47:47.689079 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-72930c95-6000-44f8-9512-0abe5189d4d3 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:47:47.690583 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-72930c95-6000-44f8-9512-0abe5189d4d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:47:47.691785 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97438dd-8c2e-48ac-83e8-cc96e0d86a3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:47:47.692747 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97438dd-8c2e-48ac-83e8-cc96e0d86a3f admin admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:47:47.693038 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97438dd-8c2e-48ac-83e8-cc96e0d86a3f admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:47:47.693316 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97438dd-8c2e-48ac-83e8-cc96e0d86a3f admin admin] PATH_INFO: `/v3/roles` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:47:47.697556 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97438dd-8c2e-48ac-83e8-cc96e0d86a3f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:47:47.697556 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97438dd-8c2e-48ac-83e8-cc96e0d86a3f admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:47:47.710362 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 789/1548] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 10:47:47 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 10:47:47.711503 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1eeab1-50dc-40a8-8928-966a3cb6869c None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:47:47.711797 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1eeab1-50dc-40a8-8928-966a3cb6869c None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:47:47.712073 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1eeab1-50dc-40a8-8928-966a3cb6869c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:47:47.715171 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 760/1549] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:47:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:47:47.716097 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72930c95-6000-44f8-9512-0abe5189d4d3 None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:47:47.729740 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72930c95-6000-44f8-9512-0abe5189d4d3 None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:47:47.740222 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb96ac6-4ad7-4388-9eb6-a4873ddf69d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:47:47.740861 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb96ac6-4ad7-4388-9eb6-a4873ddf69d9 admin admin] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:47:47.741144 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb96ac6-4ad7-4388-9eb6-a4873ddf69d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:47:47.741418 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb96ac6-4ad7-4388-9eb6-a4873ddf69d9 admin admin] PATH_INFO: `/v3/projects` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:47:47.742092 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb96ac6-4ad7-4388-9eb6-a4873ddf69d9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:47:47.744259 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb96ac6-4ad7-4388-9eb6-a4873ddf69d9 admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:47:47.751055 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-cb1eeab1-50dc-40a8-8928-966a3cb6869c None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:47:47.752584 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-cb1eeab1-50dc-40a8-8928-966a3cb6869c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:47:47.765432 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 790/1550] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:47:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:47:47.765432 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5581d1aa-569d-42ae-a5da-11d46284866f None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:47:47.765432 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5581d1aa-569d-42ae-a5da-11d46284866f None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:47:47.765432 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5581d1aa-569d-42ae-a5da-11d46284866f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:47:47.784301 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-5581d1aa-569d-42ae-a5da-11d46284866f None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:47:47.785781 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-5581d1aa-569d-42ae-a5da-11d46284866f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:47:47.788873 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 761/1551] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Jul 27 10:47:47 2022] POST /identity/v3/projects => generated 391 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 10:47:47.790179 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb96ac6-4ad7-4388-9eb6-a4873ddf69d9 admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:47:47.852563 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 791/1552] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:47:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:47:47.853990 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970667e3-1ac8-49c4-bc30-634444676000 None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:47:47.854283 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970667e3-1ac8-49c4-bc30-634444676000 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:47:47.854563 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970667e3-1ac8-49c4-bc30-634444676000 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:47:47.868632 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb96ac6-4ad7-4388-9eb6-a4873ddf69d9 admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:47:47.873870 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6e071b-27c1-4e24-9f2c-c11490dba24d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:47:47.874494 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6e071b-27c1-4e24-9f2c-c11490dba24d admin admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:47:47.874814 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6e071b-27c1-4e24-9f2c-c11490dba24d admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:47:47.875097 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6e071b-27c1-4e24-9f2c-c11490dba24d admin admin] PATH_INFO: `/v3/roles` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:47:47.875797 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6e071b-27c1-4e24-9f2c-c11490dba24d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:47:47.877262 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-970667e3-1ac8-49c4-bc30-634444676000 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:47:47.877664 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6e071b-27c1-4e24-9f2c-c11490dba24d admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:47:47.878469 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-970667e3-1ac8-49c4-bc30-634444676000 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:47:47.886679 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 792/1553] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:47:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:47:47.890591 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 762/1554] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 10:47:47 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 10:47:47.890767 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a91478a-10f4-478c-9119-8a12528f1845 None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:47:47.890767 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a91478a-10f4-478c-9119-8a12528f1845 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:47:47.890767 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a91478a-10f4-478c-9119-8a12528f1845 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:47:47.891094 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8703088e-d0f9-4e02-888d-779f668ad385 None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:47:47.891094 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8703088e-d0f9-4e02-888d-779f668ad385 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:47:47.891094 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8703088e-d0f9-4e02-888d-779f668ad385 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:47:47.908954 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-1a91478a-10f4-478c-9119-8a12528f1845 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:47:47.909169 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-8703088e-d0f9-4e02-888d-779f668ad385 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:47:47.910153 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-1a91478a-10f4-478c-9119-8a12528f1845 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:47:47.910267 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-8703088e-d0f9-4e02-888d-779f668ad385 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:47:47.918073 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 793/1555] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:47:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:47:47.918244 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 763/1556] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:47:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:47:47.919613 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a91478a-10f4-478c-9119-8a12528f1845 None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:47:47.923472 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9865af-9f1c-4e8d-96c2-04bdc2781556 None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:47:47.923767 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9865af-9f1c-4e8d-96c2-04bdc2781556 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:47:47.924050 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9865af-9f1c-4e8d-96c2-04bdc2781556 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:47:47.929322 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a91478a-10f4-478c-9119-8a12528f1845 None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:47:47.933983 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fe6c51-3d48-4ef1-8275-54d9bef462be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:47:47.934592 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fe6c51-3d48-4ef1-8275-54d9bef462be admin admin] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:47:47.935463 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fe6c51-3d48-4ef1-8275-54d9bef462be admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:47:47.935740 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fe6c51-3d48-4ef1-8275-54d9bef462be admin admin] PATH_INFO: `/v3/users` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:47:47.936412 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3fe6c51-3d48-4ef1-8275-54d9bef462be admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:47:47.938342 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3fe6c51-3d48-4ef1-8275-54d9bef462be admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:47:47.946451 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-de9865af-9f1c-4e8d-96c2-04bdc2781556 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:47:47.947691 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-de9865af-9f1c-4e8d-96c2-04bdc2781556 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:47:47.958473 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 764/1557] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:47:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:47:47.965574 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7270e4-d536-45cd-9012-212ef11a8fd9 None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:47:47.965574 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7270e4-d536-45cd-9012-212ef11a8fd9 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:47:47.965574 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7270e4-d536-45cd-9012-212ef11a8fd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:47:47.965826 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 794/1558] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Jul 27 10:47:47 2022] POST /identity/v3/users => generated 296 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 10:47:47.965915 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bd6639-e505-47ff-a277-496e0a51853c None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:47:47.966209 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bd6639-e505-47ff-a277-496e0a51853c None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:47:47.966709 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bd6639-e505-47ff-a277-496e0a51853c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:47:47.971749 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:47:47.985845 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-1e7270e4-d536-45cd-9012-212ef11a8fd9 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:47:47.987000 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-16bd6639-e505-47ff-a277-496e0a51853c None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:47:47.987137 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-1e7270e4-d536-45cd-9012-212ef11a8fd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:47:47.988183 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-16bd6639-e505-47ff-a277-496e0a51853c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:47:47.995073 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 765/1559] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:47:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:47:47.995835 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7270e4-d536-45cd-9012-212ef11a8fd9 None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:47:47.999554 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 795/1560] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:47:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:47:47.999554 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bd6639-e505-47ff-a277-496e0a51853c None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:47:48.012071 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7270e4-d536-45cd-9012-212ef11a8fd9 None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:47:48.012234 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bd6639-e505-47ff-a277-496e0a51853c None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:47:48.017287 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b6e4b6-bcc9-491a-9221-80e9494c8205 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:47:48.017443 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bab85fe-65c3-43e7-8dfa-ba7d7e49c31a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:47:48.017939 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bab85fe-65c3-43e7-8dfa-ba7d7e49c31a admin admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:47:48.018040 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b6e4b6-bcc9-491a-9221-80e9494c8205 admin admin] REQUEST_METHOD: `PUT` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:47:48.018228 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bab85fe-65c3-43e7-8dfa-ba7d7e49c31a admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:47:48.018337 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b6e4b6-bcc9-491a-9221-80e9494c8205 admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:47:48.018505 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bab85fe-65c3-43e7-8dfa-ba7d7e49c31a admin admin] PATH_INFO: `/v3/domains` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:47:48.018647 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b6e4b6-bcc9-491a-9221-80e9494c8205 admin admin] PATH_INFO: `/v3/projects/3ded08acf5604d729d22c167e59953cb/users/1c242afc942f4d889e51419710075f91/roles/4c9fc0b3baed441da2126be41cc6b04a` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:47:48.019237 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bab85fe-65c3-43e7-8dfa-ba7d7e49c31a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86828) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:47:48.019547 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bab85fe-65c3-43e7-8dfa-ba7d7e49c31a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:47:48.021164 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bab85fe-65c3-43e7-8dfa-ba7d7e49c31a admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:47:48.021678 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:47:48.034138 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 766/1561] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Jul 27 10:47:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:47:48.034878 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b6e4b6-bcc9-491a-9221-80e9494c8205 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3ded08acf5604d729d22c167e59953cb, user_id=1c242afc942f4d889e51419710075f91, role_id=4c9fc0b3baed441da2126be41cc6b04a)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:47:48.035033 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bab85fe-65c3-43e7-8dfa-ba7d7e49c31a admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:47:48.035945 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: WARNING py.warnings [None req-b0b6e4b6-bcc9-491a-9221-80e9494c8205 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 27 10:47:48.035945 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: warnings.warn(msg) Jul 27 10:47:48.035945 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: Jul 27 10:47:48.037894 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b6e4b6-bcc9-491a-9221-80e9494c8205 admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:47:48.046076 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bab85fe-65c3-43e7-8dfa-ba7d7e49c31a admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:47:48.053287 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 796/1562] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 10:47:47 2022] PUT /identity/v3/projects/3ded08acf5604d729d22c167e59953cb/users/1c242afc942f4d889e51419710075f91/roles/4c9fc0b3baed441da2126be41cc6b04a => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:47:48.054718 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3be0c6-4ad9-40b6-b8fc-8a634859cd99 None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:47:48.055012 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3be0c6-4ad9-40b6-b8fc-8a634859cd99 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:47:48.055291 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3be0c6-4ad9-40b6-b8fc-8a634859cd99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:47:48.074257 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-ae3be0c6-4ad9-40b6-b8fc-8a634859cd99 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:47:48.075531 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-ae3be0c6-4ad9-40b6-b8fc-8a634859cd99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:47:48.112148 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f0b481-29b4-49c8-bd33-57ce9a8237eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:47:48.112776 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f0b481-29b4-49c8-bd33-57ce9a8237eb admin admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:47:48.113080 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f0b481-29b4-49c8-bd33-57ce9a8237eb admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:47:48.113334 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f0b481-29b4-49c8-bd33-57ce9a8237eb admin admin] PATH_INFO: `/v3/domains` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:47:48.114029 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f0b481-29b4-49c8-bd33-57ce9a8237eb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86828) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:47:48.114343 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f0b481-29b4-49c8-bd33-57ce9a8237eb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:47:48.115991 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f0b481-29b4-49c8-bd33-57ce9a8237eb admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:47:48.129298 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 767/1563] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Jul 27 10:47:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:47:48.131145 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a02275-74e9-44d1-b708-ef1464cd1de7 None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:47:48.131618 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a02275-74e9-44d1-b708-ef1464cd1de7 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:47:48.132253 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a02275-74e9-44d1-b708-ef1464cd1de7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:47:48.136228 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 797/1564] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:47:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:47:48.137136 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3be0c6-4ad9-40b6-b8fc-8a634859cd99 None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:47:48.146331 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3be0c6-4ad9-40b6-b8fc-8a634859cd99 None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:47:48.151471 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d1eb7d-864a-4173-966b-6d4004217cac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:47:48.152189 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d1eb7d-864a-4173-966b-6d4004217cac admin admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:47:48.152479 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d1eb7d-864a-4173-966b-6d4004217cac admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:47:48.152754 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d1eb7d-864a-4173-966b-6d4004217cac admin admin] PATH_INFO: `/v3/roles` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:47:48.154099 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d1eb7d-864a-4173-966b-6d4004217cac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:47:48.154359 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-b0a02275-74e9-44d1-b708-ef1464cd1de7 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:47:48.155601 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-b0a02275-74e9-44d1-b708-ef1464cd1de7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:47:48.155813 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d1eb7d-864a-4173-966b-6d4004217cac admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:47:48.163576 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 768/1565] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:47:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:47:48.164346 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a02275-74e9-44d1-b708-ef1464cd1de7 None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:47:48.168806 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 798/1566] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 10:47:48 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 10:47:48.168806 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9518c550-2394-4840-8091-d1bc00d9ffc9 None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:47:48.168806 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9518c550-2394-4840-8091-d1bc00d9ffc9 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:47:48.168806 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9518c550-2394-4840-8091-d1bc00d9ffc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:47:48.173971 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a02275-74e9-44d1-b708-ef1464cd1de7 None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:47:48.178695 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c6ed24-395a-42ee-9d2e-43212ffa6b09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:47:48.179304 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c6ed24-395a-42ee-9d2e-43212ffa6b09 admin admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:47:48.179589 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c6ed24-395a-42ee-9d2e-43212ffa6b09 admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:47:48.179866 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c6ed24-395a-42ee-9d2e-43212ffa6b09 admin admin] PATH_INFO: `/v3/domains` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:47:48.180556 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c6ed24-395a-42ee-9d2e-43212ffa6b09 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86828) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:47:48.180889 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c6ed24-395a-42ee-9d2e-43212ffa6b09 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:47:48.182460 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c6ed24-395a-42ee-9d2e-43212ffa6b09 admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:47:48.187687 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-9518c550-2394-4840-8091-d1bc00d9ffc9 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:47:48.188920 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-9518c550-2394-4840-8091-d1bc00d9ffc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:47:48.194362 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 769/1567] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Jul 27 10:47:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:47:48.195158 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c6ed24-395a-42ee-9d2e-43212ffa6b09 admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:47:48.201159 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 799/1568] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:47:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:47:48.202506 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a720eb-1492-45fa-bc0b-7cab8e8924d8 None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:47:48.203151 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a720eb-1492-45fa-bc0b-7cab8e8924d8 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:47:48.203151 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a720eb-1492-45fa-bc0b-7cab8e8924d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:47:48.207929 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c6ed24-395a-42ee-9d2e-43212ffa6b09 admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:47:48.212863 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdd8b4c-872a-46d0-b80b-2a15892d741c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:47:48.213423 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdd8b4c-872a-46d0-b80b-2a15892d741c admin admin] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:47:48.213707 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdd8b4c-872a-46d0-b80b-2a15892d741c admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:47:48.213986 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdd8b4c-872a-46d0-b80b-2a15892d741c admin admin] PATH_INFO: `/v3/projects` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:47:48.214694 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bdd8b4c-872a-46d0-b80b-2a15892d741c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:47:48.217213 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bdd8b4c-872a-46d0-b80b-2a15892d741c admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:47:48.231099 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-e2a720eb-1492-45fa-bc0b-7cab8e8924d8 None None] MFA Rules not processed for user `1c242afc942f4d889e51419710075f91`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:47:48.232428 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-e2a720eb-1492-45fa-bc0b-7cab8e8924d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:47:48.246045 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 770/1569] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Jul 27 10:47:48 2022] POST /identity/v3/projects => generated 375 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 10:47:48.246903 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdd8b4c-872a-46d0-b80b-2a15892d741c admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:47:48.302472 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 800/1570] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:47:48 2022] POST /identity/v3/auth/tokens => generated 3222 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:47:48.303462 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a720eb-1492-45fa-bc0b-7cab8e8924d8 None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:47:48.307859 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdd8b4c-872a-46d0-b80b-2a15892d741c admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:47:48.313069 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fee0c0f-fd03-44b1-ab82-a3031aa06485 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:47:48.313695 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fee0c0f-fd03-44b1-ab82-a3031aa06485 admin admin] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:47:48.313977 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fee0c0f-fd03-44b1-ab82-a3031aa06485 admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:47:48.314231 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a720eb-1492-45fa-bc0b-7cab8e8924d8 None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:47:48.314345 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fee0c0f-fd03-44b1-ab82-a3031aa06485 admin admin] PATH_INFO: `/v3/projects` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:47:48.314934 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fee0c0f-fd03-44b1-ab82-a3031aa06485 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:47:48.316873 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fee0c0f-fd03-44b1-ab82-a3031aa06485 admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:47:48.319009 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0082f0a-b8a3-477f-bca9-a077f05e8bc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:47:48.319625 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0082f0a-b8a3-477f-bca9-a077f05e8bc3 admin admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:47:48.319901 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0082f0a-b8a3-477f-bca9-a077f05e8bc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:47:48.320177 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0082f0a-b8a3-477f-bca9-a077f05e8bc3 admin admin] PATH_INFO: `/v3/domains` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:47:48.320874 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0082f0a-b8a3-477f-bca9-a077f05e8bc3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86827) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:47:48.321190 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0082f0a-b8a3-477f-bca9-a077f05e8bc3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:47:48.322807 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0082f0a-b8a3-477f-bca9-a077f05e8bc3 admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:47:48.335047 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 801/1571] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Jul 27 10:47:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:47:48.335854 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0082f0a-b8a3-477f-bca9-a077f05e8bc3 admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:47:48.346440 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0082f0a-b8a3-477f-bca9-a077f05e8bc3 admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:47:48.350706 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 771/1572] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Jul 27 10:47:48 2022] POST /identity/v3/projects => generated 375 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 10:47:48.351092 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa830f5-de5f-4f2a-9dd3-2a6104790e77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:47:48.351529 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fee0c0f-fd03-44b1-ab82-a3031aa06485 admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:47:48.351708 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa830f5-de5f-4f2a-9dd3-2a6104790e77 admin admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:47:48.351977 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa830f5-de5f-4f2a-9dd3-2a6104790e77 admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:47:48.352260 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa830f5-de5f-4f2a-9dd3-2a6104790e77 admin admin] PATH_INFO: `/v3/domains` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:47:48.352946 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa830f5-de5f-4f2a-9dd3-2a6104790e77 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86827) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:47:48.353251 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa830f5-de5f-4f2a-9dd3-2a6104790e77 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:47:48.354999 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa830f5-de5f-4f2a-9dd3-2a6104790e77 admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:47:48.367325 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 802/1573] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Jul 27 10:47:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:47:48.368096 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa830f5-de5f-4f2a-9dd3-2a6104790e77 admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:47:48.381768 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:47:48.384422 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa830f5-de5f-4f2a-9dd3-2a6104790e77 admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:47:48.414185 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:47:48.420917 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fee0c0f-fd03-44b1-ab82-a3031aa06485 admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:47:48.425661 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91d94fa-1b33-46ee-9c0a-d0197d924b65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:47:48.426329 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91d94fa-1b33-46ee-9c0a-d0197d924b65 admin admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:47:48.426583 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91d94fa-1b33-46ee-9c0a-d0197d924b65 admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:47:48.426902 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91d94fa-1b33-46ee-9c0a-d0197d924b65 admin admin] PATH_INFO: `/v3/roles` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:47:48.427601 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91d94fa-1b33-46ee-9c0a-d0197d924b65 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:47:48.429234 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91d94fa-1b33-46ee-9c0a-d0197d924b65 admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:47:48.439903 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 772/1574] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 10:47:48 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 10:47:48.440906 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91d94fa-1b33-46ee-9c0a-d0197d924b65 admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:47:48.445388 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2b7bcc-353e-451c-bd79-23175ae7be1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:47:48.446016 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2b7bcc-353e-451c-bd79-23175ae7be1d admin admin] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:47:48.446303 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2b7bcc-353e-451c-bd79-23175ae7be1d admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:47:48.446581 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2b7bcc-353e-451c-bd79-23175ae7be1d admin admin] PATH_INFO: `/v3/projects` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:47:48.447280 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2b7bcc-353e-451c-bd79-23175ae7be1d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:47:48.448303 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: WARNING py.warnings [None req-3c2b7bcc-353e-451c-bd79-23175ae7be1d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 27 10:47:48.448303 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: warnings.warn(msg) Jul 27 10:47:48.448303 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: Jul 27 10:47:48.449504 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2b7bcc-353e-451c-bd79-23175ae7be1d admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:47:48.451312 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91d94fa-1b33-46ee-9c0a-d0197d924b65 admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:47:48.456261 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec6570a-dcbc-47f6-8074-04dc6821159f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:47:48.456902 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec6570a-dcbc-47f6-8074-04dc6821159f admin admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:47:48.457162 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec6570a-dcbc-47f6-8074-04dc6821159f admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:47:48.457438 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec6570a-dcbc-47f6-8074-04dc6821159f admin admin] PATH_INFO: `/v3/domains` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:47:48.458125 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec6570a-dcbc-47f6-8074-04dc6821159f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86828) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:47:48.458431 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec6570a-dcbc-47f6-8074-04dc6821159f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:47:48.460054 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec6570a-dcbc-47f6-8074-04dc6821159f admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:47:48.473253 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 773/1575] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Jul 27 10:47:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:47:48.473975 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec6570a-dcbc-47f6-8074-04dc6821159f admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:47:48.484079 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec6570a-dcbc-47f6-8074-04dc6821159f admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:47:48.487486 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 803/1576] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Jul 27 10:47:48 2022] POST /identity/v3/projects => generated 379 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 10:47:48.488288 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2b7bcc-353e-451c-bd79-23175ae7be1d admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:47:48.492023 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d85334-5319-4b7e-82cd-aab38e0ab513 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:47:48.492646 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d85334-5319-4b7e-82cd-aab38e0ab513 admin admin] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:47:48.492927 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d85334-5319-4b7e-82cd-aab38e0ab513 admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:47:48.493201 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d85334-5319-4b7e-82cd-aab38e0ab513 admin admin] PATH_INFO: `/v3/users` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:47:48.493897 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d85334-5319-4b7e-82cd-aab38e0ab513 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:47:48.494950 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: WARNING py.warnings [None req-44d85334-5319-4b7e-82cd-aab38e0ab513 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 27 10:47:48.494950 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: warnings.warn(msg) Jul 27 10:47:48.494950 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: Jul 27 10:47:48.496153 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d85334-5319-4b7e-82cd-aab38e0ab513 admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:47:48.518870 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 774/1577] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Jul 27 10:47:48 2022] POST /identity/v3/users => generated 288 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 10:47:48.519772 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d85334-5319-4b7e-82cd-aab38e0ab513 admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:47:48.529433 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d85334-5319-4b7e-82cd-aab38e0ab513 admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:47:48.553042 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2b7bcc-353e-451c-bd79-23175ae7be1d admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:47:48.558065 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05852d94-52ea-41e8-8164-5d5ceacbb1e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:47:48.558727 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05852d94-52ea-41e8-8164-5d5ceacbb1e3 admin admin] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:47:48.559015 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05852d94-52ea-41e8-8164-5d5ceacbb1e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:47:48.559297 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05852d94-52ea-41e8-8164-5d5ceacbb1e3 admin admin] PATH_INFO: `/v3/projects` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:47:48.560540 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05852d94-52ea-41e8-8164-5d5ceacbb1e3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:47:48.562532 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05852d94-52ea-41e8-8164-5d5ceacbb1e3 admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:47:48.588747 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc98600-6a69-432f-b4c8-a333e4ee0a99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:47:48.589413 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc98600-6a69-432f-b4c8-a333e4ee0a99 admin admin] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:47:48.589691 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc98600-6a69-432f-b4c8-a333e4ee0a99 admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:47:48.589965 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc98600-6a69-432f-b4c8-a333e4ee0a99 admin admin] PATH_INFO: `/v3/projects` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:47:48.590959 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc98600-6a69-432f-b4c8-a333e4ee0a99 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:47:48.591069 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 804/1578] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Jul 27 10:47:48 2022] POST /identity/v3/projects => generated 355 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 10:47:48.591557 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05852d94-52ea-41e8-8164-5d5ceacbb1e3 admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:47:48.592699 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc98600-6a69-432f-b4c8-a333e4ee0a99 admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:47:48.623361 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 775/1579] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Jul 27 10:47:48 2022] POST /identity/v3/projects => generated 371 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 10:47:48.623361 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc98600-6a69-432f-b4c8-a333e4ee0a99 admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:47:48.654304 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05852d94-52ea-41e8-8164-5d5ceacbb1e3 admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:47:48.680939 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc98600-6a69-432f-b4c8-a333e4ee0a99 admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:47:48.685596 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8086620-523b-4ccb-8912-57fc4cbf3bea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:47:48.686218 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8086620-523b-4ccb-8912-57fc4cbf3bea admin admin] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:47:48.686503 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8086620-523b-4ccb-8912-57fc4cbf3bea admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:47:48.686809 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8086620-523b-4ccb-8912-57fc4cbf3bea admin admin] PATH_INFO: `/v3/projects` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:47:48.687465 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8086620-523b-4ccb-8912-57fc4cbf3bea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:47:48.689375 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8086620-523b-4ccb-8912-57fc4cbf3bea admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:47:48.698475 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:47:48.709584 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:47:48.715897 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94072e96-e742-4f07-b674-4ff6219c8a8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:47:48.717136 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94072e96-e742-4f07-b674-4ff6219c8a8c admin admin] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:47:48.717492 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94072e96-e742-4f07-b674-4ff6219c8a8c admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:47:48.717839 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94072e96-e742-4f07-b674-4ff6219c8a8c admin admin] PATH_INFO: `/v3/users` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:47:48.717954 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 776/1580] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Jul 27 10:47:48 2022] POST /identity/v3/projects => generated 405 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 10:47:48.718629 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94072e96-e742-4f07-b674-4ff6219c8a8c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:47:48.718757 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8086620-523b-4ccb-8912-57fc4cbf3bea admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:47:48.720657 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94072e96-e742-4f07-b674-4ff6219c8a8c admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:47:48.741639 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 805/1581] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Jul 27 10:47:48 2022] POST /identity/v3/users => generated 288 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 10:47:48.742439 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94072e96-e742-4f07-b674-4ff6219c8a8c admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:47:48.752294 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94072e96-e742-4f07-b674-4ff6219c8a8c admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:47:48.780785 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8086620-523b-4ccb-8912-57fc4cbf3bea admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:47:48.785350 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8216fe-d724-4e91-bfbf-e8118b9d3a49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:47:48.786017 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8216fe-d724-4e91-bfbf-e8118b9d3a49 admin admin] REQUEST_METHOD: `PUT` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:47:48.786289 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8216fe-d724-4e91-bfbf-e8118b9d3a49 admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:47:48.786566 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8216fe-d724-4e91-bfbf-e8118b9d3a49 admin admin] PATH_INFO: `/v3/projects/418d6f237f034c45a7e78538b217ad5e/users/5ae9a5696bdc48778835da7a311f20c7/roles/4c9fc0b3baed441da2126be41cc6b04a` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:47:48.800680 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8216fe-d724-4e91-bfbf-e8118b9d3a49 admin admin] RBAC: Authorizing `identity:create_grant(project_id=418d6f237f034c45a7e78538b217ad5e, user_id=5ae9a5696bdc48778835da7a311f20c7, role_id=4c9fc0b3baed441da2126be41cc6b04a)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:47:48.803376 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8216fe-d724-4e91-bfbf-e8118b9d3a49 admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:47:48.809529 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea83168-a314-4750-95b5-33fab3954a80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:47:48.810976 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea83168-a314-4750-95b5-33fab3954a80 admin admin] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:47:48.811263 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea83168-a314-4750-95b5-33fab3954a80 admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:47:48.811559 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea83168-a314-4750-95b5-33fab3954a80 admin admin] PATH_INFO: `/v3/projects` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:47:48.812199 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea83168-a314-4750-95b5-33fab3954a80 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:47:48.814091 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea83168-a314-4750-95b5-33fab3954a80 admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:47:48.817400 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 777/1582] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 10:47:48 2022] PUT /identity/v3/projects/418d6f237f034c45a7e78538b217ad5e/users/5ae9a5696bdc48778835da7a311f20c7/roles/4c9fc0b3baed441da2126be41cc6b04a => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:47:48.818303 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8216fe-d724-4e91-bfbf-e8118b9d3a49 admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:47:48.843660 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 806/1583] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Jul 27 10:47:48 2022] POST /identity/v3/projects => generated 385 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 10:47:48.844480 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea83168-a314-4750-95b5-33fab3954a80 admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:47:48.884021 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8216fe-d724-4e91-bfbf-e8118b9d3a49 admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:47:48.904505 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea83168-a314-4750-95b5-33fab3954a80 admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:47:48.909144 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd577ec2-a044-402a-aa2f-0a6f62f2daea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:47:48.909760 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd577ec2-a044-402a-aa2f-0a6f62f2daea admin admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:47:48.910039 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd577ec2-a044-402a-aa2f-0a6f62f2daea admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:47:48.910311 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd577ec2-a044-402a-aa2f-0a6f62f2daea admin admin] PATH_INFO: `/v3/roles` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:47:48.911041 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd577ec2-a044-402a-aa2f-0a6f62f2daea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:47:48.912917 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd577ec2-a044-402a-aa2f-0a6f62f2daea admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:47:48.923085 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 807/1584] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 10:47:48 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 10:47:48.923864 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd577ec2-a044-402a-aa2f-0a6f62f2daea admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:47:48.933850 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd577ec2-a044-402a-aa2f-0a6f62f2daea admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:47:48.940349 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f490c229-bb9f-4fe2-8c0a-3cd3d5dd4992 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:47:48.940349 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f490c229-bb9f-4fe2-8c0a-3cd3d5dd4992 admin admin] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:47:48.940623 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5dce97f-9f31-4f38-aa5f-a52bbde4ce60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:47:48.940741 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f490c229-bb9f-4fe2-8c0a-3cd3d5dd4992 admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:47:48.940741 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f490c229-bb9f-4fe2-8c0a-3cd3d5dd4992 admin admin] PATH_INFO: `/v3/users` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:47:48.940926 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dce97f-9f31-4f38-aa5f-a52bbde4ce60 admin admin] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:47:48.940926 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dce97f-9f31-4f38-aa5f-a52bbde4ce60 admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:47:48.941089 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f490c229-bb9f-4fe2-8c0a-3cd3d5dd4992 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:47:48.941184 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dce97f-9f31-4f38-aa5f-a52bbde4ce60 admin admin] PATH_INFO: `/v3/users` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:47:48.941670 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5dce97f-9f31-4f38-aa5f-a52bbde4ce60 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:47:48.942959 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f490c229-bb9f-4fe2-8c0a-3cd3d5dd4992 admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:47:48.943638 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5dce97f-9f31-4f38-aa5f-a52bbde4ce60 admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:47:48.967648 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 778/1586] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Jul 27 10:47:48 2022] POST /identity/v3/users => generated 290 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 10:47:48.967884 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 808/1586] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Jul 27 10:47:48 2022] POST /identity/v3/users => generated 284 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 10:47:48.968519 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f490c229-bb9f-4fe2-8c0a-3cd3d5dd4992 admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:47:48.968630 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5dce97f-9f31-4f38-aa5f-a52bbde4ce60 admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:47:48.980204 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f490c229-bb9f-4fe2-8c0a-3cd3d5dd4992 admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:47:48.981019 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5dce97f-9f31-4f38-aa5f-a52bbde4ce60 admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:47:48.984894 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7911e91-d5cb-4c8c-bd95-d6fa3130a5a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:47:48.985493 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7911e91-d5cb-4c8c-bd95-d6fa3130a5a7 admin admin] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:47:48.985740 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a0baf4-c610-4092-851f-6490ef949bdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:47:48.985860 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7911e91-d5cb-4c8c-bd95-d6fa3130a5a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:47:48.986041 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7911e91-d5cb-4c8c-bd95-d6fa3130a5a7 admin admin] PATH_INFO: `/v3/users` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:47:48.986343 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a0baf4-c610-4092-851f-6490ef949bdd admin admin] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:47:48.986658 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a0baf4-c610-4092-851f-6490ef949bdd admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:47:48.986756 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7911e91-d5cb-4c8c-bd95-d6fa3130a5a7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:47:48.987059 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a0baf4-c610-4092-851f-6490ef949bdd admin admin] PATH_INFO: `/v3/users` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:47:48.987716 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a0baf4-c610-4092-851f-6490ef949bdd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:47:48.988640 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7911e91-d5cb-4c8c-bd95-d6fa3130a5a7 admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:47:48.989632 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a0baf4-c610-4092-851f-6490ef949bdd admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:47:49.007605 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:47:49.013637 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 809/1587] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Jul 27 10:47:48 2022] POST /identity/v3/users => generated 303 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 10:47:49.014861 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7911e91-d5cb-4c8c-bd95-d6fa3130a5a7 admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:47:49.017095 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 779/1588] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Jul 27 10:47:48 2022] POST /identity/v3/users => generated 286 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 10:47:49.018697 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2794cdb4-2920-42ad-9ee7-3cd4a59828fd None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:47:49.019189 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2794cdb4-2920-42ad-9ee7-3cd4a59828fd None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:47:49.019498 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2794cdb4-2920-42ad-9ee7-3cd4a59828fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:47:49.024768 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7911e91-d5cb-4c8c-bd95-d6fa3130a5a7 admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:47:49.029371 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5000365-4ac5-45df-b4b0-510f3db51b8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:47:49.029979 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5000365-4ac5-45df-b4b0-510f3db51b8c admin admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:47:49.030269 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5000365-4ac5-45df-b4b0-510f3db51b8c admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:47:49.030555 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5000365-4ac5-45df-b4b0-510f3db51b8c admin admin] PATH_INFO: `/v3/roles` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:47:49.031278 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5000365-4ac5-45df-b4b0-510f3db51b8c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:47:49.032868 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5000365-4ac5-45df-b4b0-510f3db51b8c admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:47:49.039533 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-2794cdb4-2920-42ad-9ee7-3cd4a59828fd None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:47:49.040736 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-2794cdb4-2920-42ad-9ee7-3cd4a59828fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:47:49.042887 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 810/1589] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 10:47:49 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 10:47:49.044480 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5000365-4ac5-45df-b4b0-510f3db51b8c admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:47:49.052823 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 780/1590] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:47:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:47:49.052823 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2794cdb4-2920-42ad-9ee7-3cd4a59828fd None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:47:49.053909 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5000365-4ac5-45df-b4b0-510f3db51b8c admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:47:49.058846 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253d3ad2-5599-401e-ab44-6fc0b0406cda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:47:49.060381 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253d3ad2-5599-401e-ab44-6fc0b0406cda admin admin] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:47:49.060666 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253d3ad2-5599-401e-ab44-6fc0b0406cda admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:47:49.060865 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2794cdb4-2920-42ad-9ee7-3cd4a59828fd None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:47:49.060967 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253d3ad2-5599-401e-ab44-6fc0b0406cda admin admin] PATH_INFO: `/v3/users` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:47:49.061590 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253d3ad2-5599-401e-ab44-6fc0b0406cda admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:47:49.063554 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253d3ad2-5599-401e-ab44-6fc0b0406cda admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:47:49.065436 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b38a5b-5e3c-49b8-917b-ad00062aab4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:47:49.066040 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b38a5b-5e3c-49b8-917b-ad00062aab4c admin admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:47:49.066320 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b38a5b-5e3c-49b8-917b-ad00062aab4c admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:47:49.066610 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b38a5b-5e3c-49b8-917b-ad00062aab4c admin admin] PATH_INFO: `/v3/roles` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:47:49.067319 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b38a5b-5e3c-49b8-917b-ad00062aab4c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:47:49.068897 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b38a5b-5e3c-49b8-917b-ad00062aab4c admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:47:49.074705 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=87,keystone:INSERT=13 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:47:49.085198 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 781/1591] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 10:47:49 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 10:47:49.085198 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b38a5b-5e3c-49b8-917b-ad00062aab4c admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:47:49.090418 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 811/1592] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Jul 27 10:47:49 2022] POST /identity/v3/users => generated 293 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 10:47:49.091361 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253d3ad2-5599-401e-ab44-6fc0b0406cda admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:47:49.095859 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b38a5b-5e3c-49b8-917b-ad00062aab4c admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:47:49.100529 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484704cf-6ea8-45dd-bfb8-044b57401494 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:47:49.101070 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253d3ad2-5599-401e-ab44-6fc0b0406cda admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:47:49.101202 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484704cf-6ea8-45dd-bfb8-044b57401494 admin admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:47:49.101491 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484704cf-6ea8-45dd-bfb8-044b57401494 admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:47:49.101773 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484704cf-6ea8-45dd-bfb8-044b57401494 admin admin] PATH_INFO: `/v3/roles` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:47:49.102459 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484704cf-6ea8-45dd-bfb8-044b57401494 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:47:49.104082 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484704cf-6ea8-45dd-bfb8-044b57401494 admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:47:49.105611 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d611c53-e177-4b9c-ba31-a34db8fc1c32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:47:49.106187 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d611c53-e177-4b9c-ba31-a34db8fc1c32 admin admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:47:49.106470 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d611c53-e177-4b9c-ba31-a34db8fc1c32 admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:47:49.106884 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d611c53-e177-4b9c-ba31-a34db8fc1c32 admin admin] PATH_INFO: `/v3/roles` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:47:49.107575 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d611c53-e177-4b9c-ba31-a34db8fc1c32 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:47:49.109387 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d611c53-e177-4b9c-ba31-a34db8fc1c32 admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:47:49.115055 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 782/1593] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 10:47:49 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 10:47:49.116113 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484704cf-6ea8-45dd-bfb8-044b57401494 admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:47:49.124030 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 812/1594] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 10:47:49 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 10:47:49.124839 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d611c53-e177-4b9c-ba31-a34db8fc1c32 admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:47:49.126041 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484704cf-6ea8-45dd-bfb8-044b57401494 admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:47:49.130912 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983d9cf5-6941-4804-b88d-c923d8fc19c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:47:49.131506 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983d9cf5-6941-4804-b88d-c923d8fc19c1 admin admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:47:49.131788 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983d9cf5-6941-4804-b88d-c923d8fc19c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:47:49.132088 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983d9cf5-6941-4804-b88d-c923d8fc19c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:47:49.132766 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983d9cf5-6941-4804-b88d-c923d8fc19c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:47:49.134358 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983d9cf5-6941-4804-b88d-c923d8fc19c1 admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:47:49.134871 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d611c53-e177-4b9c-ba31-a34db8fc1c32 admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:47:49.140155 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df50599a-4b53-4b38-ba71-cf265676f21b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:47:49.140756 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df50599a-4b53-4b38-ba71-cf265676f21b admin admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:47:49.141040 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df50599a-4b53-4b38-ba71-cf265676f21b admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:47:49.141319 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df50599a-4b53-4b38-ba71-cf265676f21b admin admin] PATH_INFO: `/v3/roles` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:47:49.142124 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df50599a-4b53-4b38-ba71-cf265676f21b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:47:49.143805 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df50599a-4b53-4b38-ba71-cf265676f21b admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:47:49.147351 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 783/1595] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 10:47:49 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 10:47:49.148178 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983d9cf5-6941-4804-b88d-c923d8fc19c1 admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:47:49.158183 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 813/1596] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 10:47:49 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 10:47:49.158183 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df50599a-4b53-4b38-ba71-cf265676f21b admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:47:49.158476 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983d9cf5-6941-4804-b88d-c923d8fc19c1 admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:47:49.162594 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d392820-cd39-4460-9127-d5c2d8f3552d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:47:49.163241 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d392820-cd39-4460-9127-d5c2d8f3552d admin admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:47:49.163521 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d392820-cd39-4460-9127-d5c2d8f3552d admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:47:49.163791 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d392820-cd39-4460-9127-d5c2d8f3552d admin admin] PATH_INFO: `/v3/roles` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:47:49.164475 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d392820-cd39-4460-9127-d5c2d8f3552d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:47:49.166067 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d392820-cd39-4460-9127-d5c2d8f3552d admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:47:49.176374 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 784/1597] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 10:47:49 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 10:47:49.177134 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d392820-cd39-4460-9127-d5c2d8f3552d admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:47:49.186862 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d392820-cd39-4460-9127-d5c2d8f3552d admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:47:49.191448 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc1b317-781b-4dd7-afaf-80495a82d1a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:47:49.192090 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc1b317-781b-4dd7-afaf-80495a82d1a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:47:49.192391 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc1b317-781b-4dd7-afaf-80495a82d1a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:47:49.192650 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc1b317-781b-4dd7-afaf-80495a82d1a6 admin admin] PATH_INFO: `/v3/projects/c4f3fa51a14844b1babdbd9533b7c3ec/users/9b094c00959246f5a113f600fb07215d/roles/4c9fc0b3baed441da2126be41cc6b04a` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:47:49.207391 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc1b317-781b-4dd7-afaf-80495a82d1a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c4f3fa51a14844b1babdbd9533b7c3ec, user_id=9b094c00959246f5a113f600fb07215d, role_id=4c9fc0b3baed441da2126be41cc6b04a)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:47:49.210425 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc1b317-781b-4dd7-afaf-80495a82d1a6 admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:47:49.212455 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df50599a-4b53-4b38-ba71-cf265676f21b admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:47:49.216235 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fd6df8-dd73-40c1-b2eb-9edd6d8a2bc5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7922d0a1a7c4e808d0b2a80dca0da40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:47:49.216841 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fd6df8-dd73-40c1-b2eb-9edd6d8a2bc5 service neutron] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:47:49.217125 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fd6df8-dd73-40c1-b2eb-9edd6d8a2bc5 service neutron] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:47:49.217430 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fd6df8-dd73-40c1-b2eb-9edd6d8a2bc5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:47:49.224731 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 785/1598] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 10:47:49 2022] PUT /identity/v3/projects/c4f3fa51a14844b1babdbd9533b7c3ec/users/9b094c00959246f5a113f600fb07215d/roles/4c9fc0b3baed441da2126be41cc6b04a => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:47:49.225290 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc1b317-781b-4dd7-afaf-80495a82d1a6 admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:47:49.230518 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fd6df8-dd73-40c1-b2eb-9edd6d8a2bc5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:47:49.232483 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fd6df8-dd73-40c1-b2eb-9edd6d8a2bc5 service neutron] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:47:49.242246 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 814/1599] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:47:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 10:47:49.242892 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fd6df8-dd73-40c1-b2eb-9edd6d8a2bc5 service neutron] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:47:49.251992 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fd6df8-dd73-40c1-b2eb-9edd6d8a2bc5 service neutron] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:47:49.295107 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc1b317-781b-4dd7-afaf-80495a82d1a6 admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:47:49.300186 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a51ff65-c837-470b-960e-e4564da86384 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:47:49.301175 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a51ff65-c837-470b-960e-e4564da86384 admin admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:47:49.301175 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a51ff65-c837-470b-960e-e4564da86384 admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:47:49.301412 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a51ff65-c837-470b-960e-e4564da86384 admin admin] PATH_INFO: `/v3/roles` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:47:49.302263 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a51ff65-c837-470b-960e-e4564da86384 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:47:49.303676 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a51ff65-c837-470b-960e-e4564da86384 admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:47:49.316055 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 786/1600] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 10:47:49 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 10:47:49.317346 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a51ff65-c837-470b-960e-e4564da86384 admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:47:49.323755 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82ad15a-d95b-478e-bcbe-fba3bfae530c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:47:49.324651 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82ad15a-d95b-478e-bcbe-fba3bfae530c admin admin] REQUEST_METHOD: `PUT` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:47:49.324942 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82ad15a-d95b-478e-bcbe-fba3bfae530c admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:47:49.325253 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82ad15a-d95b-478e-bcbe-fba3bfae530c admin admin] PATH_INFO: `/v3/projects/46269f939e7e47158ea3d50d0e9eee4a/users/7c9b20b42773416fbfd9a19306488933/roles/21156560eb6b436bb567a40b4976f3f4` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:47:49.329257 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a51ff65-c837-470b-960e-e4564da86384 admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:47:49.334029 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53d6b82-6ee9-4237-a410-c2b87bb6e6dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:47:49.334801 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53d6b82-6ee9-4237-a410-c2b87bb6e6dd admin admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:47:49.335009 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53d6b82-6ee9-4237-a410-c2b87bb6e6dd admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:47:49.335269 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53d6b82-6ee9-4237-a410-c2b87bb6e6dd admin admin] PATH_INFO: `/v3/roles` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:47:49.335960 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53d6b82-6ee9-4237-a410-c2b87bb6e6dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:47:49.337627 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53d6b82-6ee9-4237-a410-c2b87bb6e6dd admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:47:49.345154 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82ad15a-d95b-478e-bcbe-fba3bfae530c admin admin] RBAC: Authorizing `identity:create_grant(project_id=46269f939e7e47158ea3d50d0e9eee4a, user_id=7c9b20b42773416fbfd9a19306488933, role_id=21156560eb6b436bb567a40b4976f3f4)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:47:49.348920 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82ad15a-d95b-478e-bcbe-fba3bfae530c admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:47:49.350067 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 787/1601] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 10:47:49 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 10:47:49.351056 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53d6b82-6ee9-4237-a410-c2b87bb6e6dd admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:47:49.363729 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53d6b82-6ee9-4237-a410-c2b87bb6e6dd admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:47:49.366949 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 815/1602] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 10:47:49 2022] PUT /identity/v3/projects/46269f939e7e47158ea3d50d0e9eee4a/users/7c9b20b42773416fbfd9a19306488933/roles/21156560eb6b436bb567a40b4976f3f4 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:47:49.368206 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82ad15a-d95b-478e-bcbe-fba3bfae530c admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:47:49.444580 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff52908-8bbc-4cb2-89d7-0e3f12c965d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:47:49.445469 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff52908-8bbc-4cb2-89d7-0e3f12c965d8 admin admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:47:49.446297 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff52908-8bbc-4cb2-89d7-0e3f12c965d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:47:49.446852 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff52908-8bbc-4cb2-89d7-0e3f12c965d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:47:49.448403 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff52908-8bbc-4cb2-89d7-0e3f12c965d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:47:49.450922 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff52908-8bbc-4cb2-89d7-0e3f12c965d8 admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:47:49.451795 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82ad15a-d95b-478e-bcbe-fba3bfae530c admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:47:49.457063 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d96cfe9-b1c2-480e-b8c6-788947483fbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:47:49.457689 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d96cfe9-b1c2-480e-b8c6-788947483fbc admin admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:47:49.457971 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d96cfe9-b1c2-480e-b8c6-788947483fbc admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:47:49.458242 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d96cfe9-b1c2-480e-b8c6-788947483fbc admin admin] PATH_INFO: `/v3/roles` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:47:49.458976 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d96cfe9-b1c2-480e-b8c6-788947483fbc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:47:49.461027 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d96cfe9-b1c2-480e-b8c6-788947483fbc admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:47:49.463058 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:47:49.467970 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 788/1603] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 10:47:49 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 10:47:49.469107 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff52908-8bbc-4cb2-89d7-0e3f12c965d8 admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:47:49.477530 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 816/1604] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 10:47:49 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 10:47:49.479058 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe49a59-f4cf-4a0e-a17c-5eba2a798d51 None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:47:49.479721 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe49a59-f4cf-4a0e-a17c-5eba2a798d51 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:47:49.480062 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe49a59-f4cf-4a0e-a17c-5eba2a798d51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:47:49.487648 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff52908-8bbc-4cb2-89d7-0e3f12c965d8 admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:47:49.495973 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c26175-5c48-4481-bb09-2f5bf9d64b7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:47:49.496667 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c26175-5c48-4481-bb09-2f5bf9d64b7d admin admin] REQUEST_METHOD: `PUT` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:47:49.496956 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c26175-5c48-4481-bb09-2f5bf9d64b7d admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:47:49.497301 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c26175-5c48-4481-bb09-2f5bf9d64b7d admin admin] PATH_INFO: `/v3/projects/66c9954b29af4eb5a8ca587e290e5c22/users/6fb6cab163434ef6a1e909e86520c233/roles/4c9fc0b3baed441da2126be41cc6b04a` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:47:49.500042 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:47:49.503153 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-bfe49a59-f4cf-4a0e-a17c-5eba2a798d51 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:47:49.504932 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-bfe49a59-f4cf-4a0e-a17c-5eba2a798d51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:47:49.519329 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 817/1605] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:47:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:47:49.520720 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe49a59-f4cf-4a0e-a17c-5eba2a798d51 None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:47:49.523712 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c26175-5c48-4481-bb09-2f5bf9d64b7d admin admin] RBAC: Authorizing `identity:create_grant(project_id=66c9954b29af4eb5a8ca587e290e5c22, user_id=6fb6cab163434ef6a1e909e86520c233, role_id=4c9fc0b3baed441da2126be41cc6b04a)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:47:49.529555 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c26175-5c48-4481-bb09-2f5bf9d64b7d admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:47:49.532672 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe49a59-f4cf-4a0e-a17c-5eba2a798d51 None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:47:49.538824 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd3bc48-4741-4bb2-952b-e405f6278378 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:47:49.539539 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd3bc48-4741-4bb2-952b-e405f6278378 admin admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:47:49.539898 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd3bc48-4741-4bb2-952b-e405f6278378 admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:47:49.540278 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd3bc48-4741-4bb2-952b-e405f6278378 admin admin] PATH_INFO: `/v3/roles` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:47:49.541065 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd3bc48-4741-4bb2-952b-e405f6278378 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:47:49.542874 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd3bc48-4741-4bb2-952b-e405f6278378 admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:47:49.548497 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 789/1606] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 10:47:49 2022] PUT /identity/v3/projects/66c9954b29af4eb5a8ca587e290e5c22/users/6fb6cab163434ef6a1e909e86520c233/roles/4c9fc0b3baed441da2126be41cc6b04a => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:47:49.550271 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c26175-5c48-4481-bb09-2f5bf9d64b7d admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:47:49.555417 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 818/1607] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 10:47:49 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 10:47:49.556459 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd3bc48-4741-4bb2-952b-e405f6278378 admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:47:49.568683 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd3bc48-4741-4bb2-952b-e405f6278378 admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:47:49.642386 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e59010-8916-4447-a19a-66d3a5a63dff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:47:49.643121 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e59010-8916-4447-a19a-66d3a5a63dff admin admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:47:49.643655 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e59010-8916-4447-a19a-66d3a5a63dff admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:47:49.644158 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e59010-8916-4447-a19a-66d3a5a63dff admin admin] PATH_INFO: `/v3/roles` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:47:49.644626 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c26175-5c48-4481-bb09-2f5bf9d64b7d admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:47:49.645014 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e59010-8916-4447-a19a-66d3a5a63dff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:47:49.646980 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e59010-8916-4447-a19a-66d3a5a63dff admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:47:49.651655 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acec4fb-c2c4-4a1f-bcae-102a872f5bec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:47:49.652367 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acec4fb-c2c4-4a1f-bcae-102a872f5bec admin admin] REQUEST_METHOD: `PUT` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:47:49.652655 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acec4fb-c2c4-4a1f-bcae-102a872f5bec admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:47:49.652937 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acec4fb-c2c4-4a1f-bcae-102a872f5bec admin admin] PATH_INFO: `/v3/projects/728ed167367f4d99ad27e0b98c969e1a/users/b82963d0f8f245aaa295379a38908c4d/roles/4c9fc0b3baed441da2126be41cc6b04a` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:47:49.664090 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 819/1608] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 10:47:49 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 10:47:49.665531 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e59010-8916-4447-a19a-66d3a5a63dff admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:47:49.675046 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6acec4fb-c2c4-4a1f-bcae-102a872f5bec admin admin] RBAC: Authorizing `identity:create_grant(project_id=728ed167367f4d99ad27e0b98c969e1a, user_id=b82963d0f8f245aaa295379a38908c4d, role_id=4c9fc0b3baed441da2126be41cc6b04a)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:47:49.677178 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e59010-8916-4447-a19a-66d3a5a63dff admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:47:49.678681 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6acec4fb-c2c4-4a1f-bcae-102a872f5bec admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:47:49.683333 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befd6eab-deac-4f04-b85a-0945ec6bacc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:47:49.684151 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befd6eab-deac-4f04-b85a-0945ec6bacc5 admin admin] REQUEST_METHOD: `PUT` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:47:49.685205 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befd6eab-deac-4f04-b85a-0945ec6bacc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:47:49.685840 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befd6eab-deac-4f04-b85a-0945ec6bacc5 admin admin] PATH_INFO: `/v3/projects/977cf028e9a1422f96c2088a4db3d8d6/users/798bfb641ab144b28ad904bb8ed6f928/roles/4c9fc0b3baed441da2126be41cc6b04a` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:47:49.698511 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 790/1609] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 10:47:49 2022] PUT /identity/v3/projects/728ed167367f4d99ad27e0b98c969e1a/users/b82963d0f8f245aaa295379a38908c4d/roles/4c9fc0b3baed441da2126be41cc6b04a => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:47:49.700144 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acec4fb-c2c4-4a1f-bcae-102a872f5bec admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:47:49.710351 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befd6eab-deac-4f04-b85a-0945ec6bacc5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=977cf028e9a1422f96c2088a4db3d8d6, user_id=798bfb641ab144b28ad904bb8ed6f928, role_id=4c9fc0b3baed441da2126be41cc6b04a)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:47:49.713366 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befd6eab-deac-4f04-b85a-0945ec6bacc5 admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:47:49.735505 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 820/1610] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 10:47:49 2022] PUT /identity/v3/projects/977cf028e9a1422f96c2088a4db3d8d6/users/798bfb641ab144b28ad904bb8ed6f928/roles/4c9fc0b3baed441da2126be41cc6b04a => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:47:49.736843 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befd6eab-deac-4f04-b85a-0945ec6bacc5 admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:47:49.738872 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-befd6eab-deac-4f04-b85a-0945ec6bacc5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:47:49.796624 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acec4fb-c2c4-4a1f-bcae-102a872f5bec admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:47:49.805133 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befd6eab-deac-4f04-b85a-0945ec6bacc5 admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:47:49.810438 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5c9615-90d1-49b8-9a07-98d933008564 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7922d0a1a7c4e808d0b2a80dca0da40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:47:49.811145 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5c9615-90d1-49b8-9a07-98d933008564 service neutron] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:47:49.811574 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5c9615-90d1-49b8-9a07-98d933008564 service neutron] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:47:49.811753 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5c9615-90d1-49b8-9a07-98d933008564 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:47:49.870205 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ee10a3-c374-4c88-bc24-ea7213e308bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:47:49.870920 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ee10a3-c374-4c88-bc24-ea7213e308bd admin admin] REQUEST_METHOD: `PUT` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:47:49.871210 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ee10a3-c374-4c88-bc24-ea7213e308bd admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:47:49.871490 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ee10a3-c374-4c88-bc24-ea7213e308bd admin admin] PATH_INFO: `/v3/projects/cb55b64bad614375827066b02150d6bb/users/c38f9d253f024686ba765c6a7fec5b4b/roles/4c9fc0b3baed441da2126be41cc6b04a` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:47:49.881338 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5c9615-90d1-49b8-9a07-98d933008564 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:47:49.882896 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5c9615-90d1-49b8-9a07-98d933008564 service neutron] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:47:49.888912 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:47:49.889062 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ee10a3-c374-4c88-bc24-ea7213e308bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb55b64bad614375827066b02150d6bb, user_id=c38f9d253f024686ba765c6a7fec5b4b, role_id=4c9fc0b3baed441da2126be41cc6b04a)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:47:49.892220 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ee10a3-c374-4c88-bc24-ea7213e308bd admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:47:49.895884 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:47:49.902294 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 821/1611] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:47:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 10:47:49.904439 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5791348-c019-42af-bc57-00b232fb3a63 None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:47:49.904439 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5791348-c019-42af-bc57-00b232fb3a63 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:47:49.905046 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5791348-c019-42af-bc57-00b232fb3a63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:47:49.910154 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 791/1612] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 10:47:49 2022] PUT /identity/v3/projects/cb55b64bad614375827066b02150d6bb/users/c38f9d253f024686ba765c6a7fec5b4b/roles/4c9fc0b3baed441da2126be41cc6b04a => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:47:49.911771 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ee10a3-c374-4c88-bc24-ea7213e308bd admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:47:49.949662 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-e5791348-c019-42af-bc57-00b232fb3a63 None None] MFA Rules not processed for user `6fb6cab163434ef6a1e909e86520c233`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:47:49.951852 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-e5791348-c019-42af-bc57-00b232fb3a63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:47:50.002749 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ee10a3-c374-4c88-bc24-ea7213e308bd admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:47:50.010344 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648015fc-a8d9-453f-a345-c80150d351de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:47:50.011091 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648015fc-a8d9-453f-a345-c80150d351de admin admin] REQUEST_METHOD: `PUT` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:47:50.011967 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648015fc-a8d9-453f-a345-c80150d351de admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:47:50.011967 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648015fc-a8d9-453f-a345-c80150d351de admin admin] PATH_INFO: `/v3/projects/e52022826ef84644b89cca2931ac53a1/users/5add2ee95b6c47a783898368f1bf66d0/roles/4c9fc0b3baed441da2126be41cc6b04a` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:47:50.033719 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648015fc-a8d9-453f-a345-c80150d351de admin admin] RBAC: Authorizing `identity:create_grant(project_id=e52022826ef84644b89cca2931ac53a1, user_id=5add2ee95b6c47a783898368f1bf66d0, role_id=4c9fc0b3baed441da2126be41cc6b04a)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:47:50.037206 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648015fc-a8d9-453f-a345-c80150d351de admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:47:50.050328 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 822/1613] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:47:49 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:47:50.051790 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5791348-c019-42af-bc57-00b232fb3a63 None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:47:50.056755 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 792/1614] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 10:47:49 2022] PUT /identity/v3/projects/e52022826ef84644b89cca2931ac53a1/users/5add2ee95b6c47a783898368f1bf66d0/roles/4c9fc0b3baed441da2126be41cc6b04a => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:47:50.058276 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11290f7c-4bc2-4702-9da7-d9b76c682dca None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:47:50.058703 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11290f7c-4bc2-4702-9da7-d9b76c682dca None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:47:50.059064 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11290f7c-4bc2-4702-9da7-d9b76c682dca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:47:50.074876 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5791348-c019-42af-bc57-00b232fb3a63 None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:47:50.097193 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-11290f7c-4bc2-4702-9da7-d9b76c682dca None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:47:50.099073 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-11290f7c-4bc2-4702-9da7-d9b76c682dca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:47:50.150483 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e85936f-b229-467a-ad5e-a3b28932bf1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:47:50.152050 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e85936f-b229-467a-ad5e-a3b28932bf1f admin admin] REQUEST_METHOD: `PUT` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:47:50.152608 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e85936f-b229-467a-ad5e-a3b28932bf1f admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:47:50.153547 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e85936f-b229-467a-ad5e-a3b28932bf1f admin admin] PATH_INFO: `/v3/projects/46269f939e7e47158ea3d50d0e9eee4a/users/7c9b20b42773416fbfd9a19306488933/roles/21156560eb6b436bb567a40b4976f3f4` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:47:50.157357 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e85936f-b229-467a-ad5e-a3b28932bf1f admin admin] RBAC: Authorizing `identity:create_grant(project_id=46269f939e7e47158ea3d50d0e9eee4a, user_id=7c9b20b42773416fbfd9a19306488933, role_id=21156560eb6b436bb567a40b4976f3f4)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:47:50.160859 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e85936f-b229-467a-ad5e-a3b28932bf1f admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:47:50.178675 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 823/1615] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 10:47:50 2022] PUT /identity/v3/projects/46269f939e7e47158ea3d50d0e9eee4a/users/7c9b20b42773416fbfd9a19306488933/roles/21156560eb6b436bb567a40b4976f3f4 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:47:50.180494 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74e2832-f4ee-4633-98f9-4d2d0408c43c None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:47:50.180984 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74e2832-f4ee-4633-98f9-4d2d0408c43c None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:47:50.181648 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74e2832-f4ee-4633-98f9-4d2d0408c43c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:47:50.210258 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 793/1616] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:47:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:47:50.212046 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26a6d55-9fbd-439a-91b1-1352297e4a55 None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:47:50.212389 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26a6d55-9fbd-439a-91b1-1352297e4a55 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:47:50.212708 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26a6d55-9fbd-439a-91b1-1352297e4a55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:47:50.214891 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-b74e2832-f4ee-4633-98f9-4d2d0408c43c None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:47:50.216913 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-b74e2832-f4ee-4633-98f9-4d2d0408c43c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:47:50.264763 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-d26a6d55-9fbd-439a-91b1-1352297e4a55 None None] MFA Rules not processed for user `c38f9d253f024686ba765c6a7fec5b4b`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:47:50.266834 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-d26a6d55-9fbd-439a-91b1-1352297e4a55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:47:50.296481 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 824/1617] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:47:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:47:50.297861 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35f0a7c-e236-44ef-bb01-82efc01ed6f6 None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:47:50.298237 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35f0a7c-e236-44ef-bb01-82efc01ed6f6 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:47:50.298580 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35f0a7c-e236-44ef-bb01-82efc01ed6f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:47:50.358543 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-b35f0a7c-e236-44ef-bb01-82efc01ed6f6 None None] MFA Rules not processed for user `5add2ee95b6c47a783898368f1bf66d0`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:47:50.359700 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-b35f0a7c-e236-44ef-bb01-82efc01ed6f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:47:50.379336 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:47:50.386043 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 794/1618] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:47:50 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:47:50.389341 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a999329-2b12-4d34-9445-6d320df19c11 None None] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:47:50.389892 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a999329-2b12-4d34-9445-6d320df19c11 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:47:50.390371 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a999329-2b12-4d34-9445-6d320df19c11 None None] PATH_INFO: `/` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:47:50.393208 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 795/1619] 10.222.0.36 () {54 vars in 962 bytes} [Wed Jul 27 10:47:50 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 10:47:50.400482 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7f5131-a718-4b8d-bd24-f2b52c24a5cb None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:47:50.401047 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7f5131-a718-4b8d-bd24-f2b52c24a5cb None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:47:50.402771 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7f5131-a718-4b8d-bd24-f2b52c24a5cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:47:50.427638 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:47:50.440058 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-fa7f5131-a718-4b8d-bd24-f2b52c24a5cb None None] MFA Rules not processed for user `7c9b20b42773416fbfd9a19306488933`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:47:50.442102 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-fa7f5131-a718-4b8d-bd24-f2b52c24a5cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:47:50.508176 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 825/1620] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:47:50 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:47:50.509570 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35f0a7c-e236-44ef-bb01-82efc01ed6f6 None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:47:50.593523 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 796/1621] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:47:50 2022] POST /identity/v3/auth/tokens => generated 3289 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:47:50.594552 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7f5131-a718-4b8d-bd24-f2b52c24a5cb None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:47:50.606270 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35f0a7c-e236-44ef-bb01-82efc01ed6f6 None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:47:50.611345 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b572fe71-365f-4a85-b6ae-8e3770aaa215 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7922d0a1a7c4e808d0b2a80dca0da40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:47:50.612289 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b572fe71-365f-4a85-b6ae-8e3770aaa215 service neutron] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:47:50.613250 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b572fe71-365f-4a85-b6ae-8e3770aaa215 service neutron] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:47:50.613774 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b572fe71-365f-4a85-b6ae-8e3770aaa215 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:47:50.618350 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7f5131-a718-4b8d-bd24-f2b52c24a5cb None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:47:50.625977 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34f1e57-f1d9-419c-aef3-7bbdbbb463c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7922d0a1a7c4e808d0b2a80dca0da40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:47:50.627469 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34f1e57-f1d9-419c-aef3-7bbdbbb463c1 service neutron] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:47:50.628037 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34f1e57-f1d9-419c-aef3-7bbdbbb463c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:47:50.628516 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b572fe71-365f-4a85-b6ae-8e3770aaa215 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:47:50.628712 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34f1e57-f1d9-419c-aef3-7bbdbbb463c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:47:50.630728 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b572fe71-365f-4a85-b6ae-8e3770aaa215 service neutron] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:47:50.643042 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 826/1622] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:47:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 10:47:50.644163 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b572fe71-365f-4a85-b6ae-8e3770aaa215 service neutron] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:47:50.654720 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34f1e57-f1d9-419c-aef3-7bbdbbb463c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:47:50.656138 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b572fe71-365f-4a85-b6ae-8e3770aaa215 service neutron] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:47:50.656815 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34f1e57-f1d9-419c-aef3-7bbdbbb463c1 service neutron] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:47:50.661580 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bb409a-29de-47b8-a6ce-1a0691a10ffe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:47:50.662230 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bb409a-29de-47b8-a6ce-1a0691a10ffe admin admin] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:47:50.662632 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bb409a-29de-47b8-a6ce-1a0691a10ffe admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:47:50.663022 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bb409a-29de-47b8-a6ce-1a0691a10ffe admin admin] PATH_INFO: `/v3/projects` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:47:50.663969 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07bb409a-29de-47b8-a6ce-1a0691a10ffe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:47:50.666283 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07bb409a-29de-47b8-a6ce-1a0691a10ffe admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:47:50.682520 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 797/1623] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:47:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 10:47:50.683913 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795ec0eb-b255-4bb2-9ecf-c38acefacf62 None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:47:50.684208 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795ec0eb-b255-4bb2-9ecf-c38acefacf62 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:47:50.686822 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795ec0eb-b255-4bb2-9ecf-c38acefacf62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:47:50.702319 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 827/1624] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Jul 27 10:47:50 2022] POST /identity/v3/projects => generated 373 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 10:47:50.708470 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113f6c14-9e08-4d33-a641-768a750fef7e None None] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:47:50.708923 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113f6c14-9e08-4d33-a641-768a750fef7e None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:47:50.709586 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113f6c14-9e08-4d33-a641-768a750fef7e None None] PATH_INFO: `/` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:47:50.713971 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 828/1625] 10.222.0.36 () {54 vars in 962 bytes} [Wed Jul 27 10:47:50 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 10:47:50.715454 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113f6c14-9e08-4d33-a641-768a750fef7e None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:47:50.775716 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-795ec0eb-b255-4bb2-9ecf-c38acefacf62 None None] MFA Rules not processed for user `e059d8b6c28c4fcba925d05d7c98e4ac`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:47:50.777042 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-795ec0eb-b255-4bb2-9ecf-c38acefacf62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:47:50.822383 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113f6c14-9e08-4d33-a641-768a750fef7e None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:47:50.829019 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ce447a-257d-4975-ac6d-df38e1e81e7d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '864d35f268cf45148756109759bb2743', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:47:50.830300 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ce447a-257d-4975-ac6d-df38e1e81e7d service cinder] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:47:50.830300 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ce447a-257d-4975-ac6d-df38e1e81e7d service cinder] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:47:50.832907 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ce447a-257d-4975-ac6d-df38e1e81e7d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:47:50.915068 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:47:50.933586 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 798/1626] 10.222.0.36 () {58 vars in 1059 bytes} [Wed Jul 27 10:47:50 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:47:50.934449 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795ec0eb-b255-4bb2-9ecf-c38acefacf62 None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:47:50.945023 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ce447a-257d-4975-ac6d-df38e1e81e7d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:47:50.946901 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ce447a-257d-4975-ac6d-df38e1e81e7d service cinder] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:47:50.978752 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 829/1627] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:47:50 2022] GET /identity/v3/auth/tokens => generated 3289 bytes in 264 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 10:47:50.983682 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05604a71-6917-4dfb-8989-33e207d0619a None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:47:50.984078 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05604a71-6917-4dfb-8989-33e207d0619a None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:47:50.984370 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05604a71-6917-4dfb-8989-33e207d0619a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:47:51.051566 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-05604a71-6917-4dfb-8989-33e207d0619a None None] MFA Rules not processed for user `e059d8b6c28c4fcba925d05d7c98e4ac`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:47:51.057674 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-05604a71-6917-4dfb-8989-33e207d0619a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:47:51.073559 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795ec0eb-b255-4bb2-9ecf-c38acefacf62 None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:47:51.089550 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2335fd-b5ac-42f2-bf4a-8ed3092c01b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:47:51.090741 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2335fd-b5ac-42f2-bf4a-8ed3092c01b2 admin admin] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:47:51.091081 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2335fd-b5ac-42f2-bf4a-8ed3092c01b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:47:51.091360 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2335fd-b5ac-42f2-bf4a-8ed3092c01b2 admin admin] PATH_INFO: `/v3/users` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:47:51.092035 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2335fd-b5ac-42f2-bf4a-8ed3092c01b2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:47:51.093985 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2335fd-b5ac-42f2-bf4a-8ed3092c01b2 admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:47:51.097148 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 830/1628] 10.222.0.36 () {58 vars in 1059 bytes} [Wed Jul 27 10:47:50 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:47:51.098196 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05604a71-6917-4dfb-8989-33e207d0619a None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:47:51.112445 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:47:51.119935 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05604a71-6917-4dfb-8989-33e207d0619a None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:47:51.142074 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c82993-7c3a-4df4-8bff-08dbc334031f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e059d8b6c28c4fcba925d05d7c98e4ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:47:51.143288 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c82993-7c3a-4df4-8bff-08dbc334031f service nova] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:47:51.144770 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c82993-7c3a-4df4-8bff-08dbc334031f service nova] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:47:51.148066 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 799/1629] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Jul 27 10:47:50 2022] POST /identity/v3/users => generated 293 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 10:47:51.148342 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c82993-7c3a-4df4-8bff-08dbc334031f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:47:51.148534 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2335fd-b5ac-42f2-bf4a-8ed3092c01b2 admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:47:51.175222 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2335fd-b5ac-42f2-bf4a-8ed3092c01b2 admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:47:51.190278 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9c16d1-13c4-44ab-b2cc-1b670a8c02e9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e059d8b6c28c4fcba925d05d7c98e4ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:47:51.191549 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9c16d1-13c4-44ab-b2cc-1b670a8c02e9 service nova] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:47:51.191851 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9c16d1-13c4-44ab-b2cc-1b670a8c02e9 service nova] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:47:51.192247 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9c16d1-13c4-44ab-b2cc-1b670a8c02e9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:47:51.260671 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c82993-7c3a-4df4-8bff-08dbc334031f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:47:51.262497 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c82993-7c3a-4df4-8bff-08dbc334031f service nova] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:47:51.274511 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 831/1630] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:47:51 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 10:47:51.275505 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c82993-7c3a-4df4-8bff-08dbc334031f service nova] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:47:51.286906 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c82993-7c3a-4df4-8bff-08dbc334031f service nova] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:47:51.291557 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71bff20-b573-4ab2-9ed6-a26310a1836e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '864d35f268cf45148756109759bb2743', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:47:51.292181 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71bff20-b573-4ab2-9ed6-a26310a1836e service cinder] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:47:51.292511 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71bff20-b573-4ab2-9ed6-a26310a1836e service cinder] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:47:51.292989 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71bff20-b573-4ab2-9ed6-a26310a1836e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:47:51.364185 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9c16d1-13c4-44ab-b2cc-1b670a8c02e9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:47:51.365534 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9c16d1-13c4-44ab-b2cc-1b670a8c02e9 service nova] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:47:51.400851 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71bff20-b573-4ab2-9ed6-a26310a1836e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:47:51.402148 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 800/1631] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:47:51 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 256 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 10:47:51.402644 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71bff20-b573-4ab2-9ed6-a26310a1836e service cinder] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:47:51.404082 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9c16d1-13c4-44ab-b2cc-1b670a8c02e9 service nova] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:47:51.414911 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 832/1632] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:47:51 2022] GET /identity/v3/auth/tokens => generated 3222 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 10:47:51.462942 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9c16d1-13c4-44ab-b2cc-1b670a8c02e9 service nova] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:47:51.487390 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93687604-39ab-418e-9fe9-b741aa26a865 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:47:51.488046 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93687604-39ab-418e-9fe9-b741aa26a865 admin admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:47:51.488336 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93687604-39ab-418e-9fe9-b741aa26a865 admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:47:51.488616 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93687604-39ab-418e-9fe9-b741aa26a865 admin admin] PATH_INFO: `/v3/roles` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:47:51.489324 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93687604-39ab-418e-9fe9-b741aa26a865 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:47:51.490993 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93687604-39ab-418e-9fe9-b741aa26a865 admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:47:51.540541 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 801/1633] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 10:47:51 2022] GET /identity/v3/roles => generated 1420 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 10:47:51.541408 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71bff20-b573-4ab2-9ed6-a26310a1836e service cinder] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:47:51.562638 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71bff20-b573-4ab2-9ed6-a26310a1836e service cinder] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:47:51.576535 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03878793-8c46-4375-86a0-f0e47bbfaaec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:47:51.577715 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03878793-8c46-4375-86a0-f0e47bbfaaec admin admin] REQUEST_METHOD: `PUT` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:47:51.578278 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03878793-8c46-4375-86a0-f0e47bbfaaec admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:47:51.578878 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03878793-8c46-4375-86a0-f0e47bbfaaec admin admin] PATH_INFO: `/v3/projects/6edec309c42a4ab080f9dc7bc061aefe/users/bfcc4bf290e442de848d0d07a4119e01/roles/21156560eb6b436bb567a40b4976f3f4` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:47:51.633915 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03878793-8c46-4375-86a0-f0e47bbfaaec admin admin] RBAC: Authorizing `identity:create_grant(project_id=6edec309c42a4ab080f9dc7bc061aefe, user_id=bfcc4bf290e442de848d0d07a4119e01, role_id=21156560eb6b436bb567a40b4976f3f4)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:47:51.645552 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03878793-8c46-4375-86a0-f0e47bbfaaec admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:47:51.701627 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 833/1634] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 10:47:51 2022] PUT /identity/v3/projects/6edec309c42a4ab080f9dc7bc061aefe/users/bfcc4bf290e442de848d0d07a4119e01/roles/21156560eb6b436bb567a40b4976f3f4 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:47:51.702780 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93687604-39ab-418e-9fe9-b741aa26a865 admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:47:51.775738 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93687604-39ab-418e-9fe9-b741aa26a865 admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:47:51.867141 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc9804f-63cb-4eb5-8da2-b2082b70c0c8 None None] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:47:51.867141 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc9804f-63cb-4eb5-8da2-b2082b70c0c8 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:47:51.867141 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc9804f-63cb-4eb5-8da2-b2082b70c0c8 None None] PATH_INFO: `/` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:47:51.868643 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 834/1635] 10.222.0.36 () {54 vars in 912 bytes} [Wed Jul 27 10:47:51 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 10:47:51.876382 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb0eb48-f158-4836-8df2-a62a16969502 None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:47:51.876850 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb0eb48-f158-4836-8df2-a62a16969502 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:47:51.877393 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb0eb48-f158-4836-8df2-a62a16969502 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:47:51.917888 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-7eb0eb48-f158-4836-8df2-a62a16969502 None None] MFA Rules not processed for user `e059d8b6c28c4fcba925d05d7c98e4ac`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:47:51.919639 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-7eb0eb48-f158-4836-8df2-a62a16969502 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:47:51.947131 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7106d683-de46-4ad7-b2c5-cd51e5045297 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:47:51.958077 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7106d683-de46-4ad7-b2c5-cd51e5045297 admin admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:47:51.961782 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7106d683-de46-4ad7-b2c5-cd51e5045297 admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:47:51.962398 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7106d683-de46-4ad7-b2c5-cd51e5045297 admin admin] PATH_INFO: `/v3/roles` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:47:51.968284 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7106d683-de46-4ad7-b2c5-cd51e5045297 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:47:51.977507 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7106d683-de46-4ad7-b2c5-cd51e5045297 admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:47:52.027452 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 802/1636] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 10:47:51 2022] GET /identity/v3/roles => generated 1420 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 10:47:52.034462 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7106d683-de46-4ad7-b2c5-cd51e5045297 admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:47:52.074739 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:47:52.076604 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7106d683-de46-4ad7-b2c5-cd51e5045297 admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:47:52.085715 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff198dcb-b8b0-4b18-a2fb-2243e3652eea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:47:52.087109 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff198dcb-b8b0-4b18-a2fb-2243e3652eea admin admin] REQUEST_METHOD: `PUT` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:47:52.087109 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff198dcb-b8b0-4b18-a2fb-2243e3652eea admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:47:52.087450 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff198dcb-b8b0-4b18-a2fb-2243e3652eea admin admin] PATH_INFO: `/v3/projects/6edec309c42a4ab080f9dc7bc061aefe/users/bfcc4bf290e442de848d0d07a4119e01/roles/21156560eb6b436bb567a40b4976f3f4` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:47:52.091071 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff198dcb-b8b0-4b18-a2fb-2243e3652eea admin admin] RBAC: Authorizing `identity:create_grant(project_id=6edec309c42a4ab080f9dc7bc061aefe, user_id=bfcc4bf290e442de848d0d07a4119e01, role_id=21156560eb6b436bb567a40b4976f3f4)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:47:52.094744 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff198dcb-b8b0-4b18-a2fb-2243e3652eea admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:47:52.117563 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 803/1637] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 10:47:52 2022] PUT /identity/v3/projects/6edec309c42a4ab080f9dc7bc061aefe/users/bfcc4bf290e442de848d0d07a4119e01/roles/21156560eb6b436bb567a40b4976f3f4 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:47:52.127707 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a488bd-57af-4245-9dd8-c98b38ad8cfb None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:47:52.128288 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a488bd-57af-4245-9dd8-c98b38ad8cfb None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:47:52.128651 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a488bd-57af-4245-9dd8-c98b38ad8cfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:47:52.131013 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 835/1638] 10.222.0.36 () {58 vars in 1009 bytes} [Wed Jul 27 10:47:51 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:47:52.132483 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e025e53d-2eac-4347-acaa-4fc8ffc8c2bc None None] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:47:52.132833 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e025e53d-2eac-4347-acaa-4fc8ffc8c2bc None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:47:52.133179 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e025e53d-2eac-4347-acaa-4fc8ffc8c2bc None None] PATH_INFO: `/` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:47:52.135030 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 836/1639] 10.222.0.36 () {54 vars in 912 bytes} [Wed Jul 27 10:47:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 10:47:52.146962 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a111ad0d-f5e3-4e66-848e-fe4321bc9ac0 None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:47:52.147527 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a111ad0d-f5e3-4e66-848e-fe4321bc9ac0 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:47:52.147934 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a111ad0d-f5e3-4e66-848e-fe4321bc9ac0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:47:52.164769 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-b4a488bd-57af-4245-9dd8-c98b38ad8cfb None None] MFA Rules not processed for user `bfcc4bf290e442de848d0d07a4119e01`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:47:52.168231 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-b4a488bd-57af-4245-9dd8-c98b38ad8cfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:47:52.180855 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-a111ad0d-f5e3-4e66-848e-fe4321bc9ac0 None None] MFA Rules not processed for user `e059d8b6c28c4fcba925d05d7c98e4ac`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:47:52.182297 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-a111ad0d-f5e3-4e66-848e-fe4321bc9ac0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:47:52.247723 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:47:52.295091 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 804/1640] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:47:52 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:47:52.296386 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a488bd-57af-4245-9dd8-c98b38ad8cfb None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:47:52.300333 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 837/1641] 10.222.0.36 () {58 vars in 1009 bytes} [Wed Jul 27 10:47:52 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:47:52.307961 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a111ad0d-f5e3-4e66-848e-fe4321bc9ac0 None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:47:52.405415 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a488bd-57af-4245-9dd8-c98b38ad8cfb None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:47:52.418423 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ac83d1-43dd-4d3c-a9cf-f6230a0b360e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '864d35f268cf45148756109759bb2743', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:47:52.419469 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ac83d1-43dd-4d3c-a9cf-f6230a0b360e service cinder] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:47:52.419934 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ac83d1-43dd-4d3c-a9cf-f6230a0b360e service cinder] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:47:52.420200 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a111ad0d-f5e3-4e66-848e-fe4321bc9ac0 None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:47:52.421710 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ac83d1-43dd-4d3c-a9cf-f6230a0b360e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:47:52.433445 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f80cd34-4630-4bd6-bde3-d96d7d2ea577 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '864d35f268cf45148756109759bb2743', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:47:52.434139 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f80cd34-4630-4bd6-bde3-d96d7d2ea577 service cinder] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:47:52.434480 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f80cd34-4630-4bd6-bde3-d96d7d2ea577 service cinder] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:47:52.434903 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f80cd34-4630-4bd6-bde3-d96d7d2ea577 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:47:52.447738 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ac83d1-43dd-4d3c-a9cf-f6230a0b360e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:47:52.449409 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ac83d1-43dd-4d3c-a9cf-f6230a0b360e service cinder] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:47:52.465359 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f80cd34-4630-4bd6-bde3-d96d7d2ea577 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:47:52.468474 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f80cd34-4630-4bd6-bde3-d96d7d2ea577 service cinder] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:47:52.486813 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 805/1642] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:47:52 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 10:47:52.515526 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 838/1643] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:47:52 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 10:47:52.619127 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ac83d1-43dd-4d3c-a9cf-f6230a0b360e service cinder] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:47:52.636737 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ac83d1-43dd-4d3c-a9cf-f6230a0b360e service cinder] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:47:52.647684 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8594274-e392-433f-be44-bec8625635c6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '864d35f268cf45148756109759bb2743', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:47:52.648592 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8594274-e392-433f-be44-bec8625635c6 service cinder] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:47:52.649119 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8594274-e392-433f-be44-bec8625635c6 service cinder] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:47:52.650277 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8594274-e392-433f-be44-bec8625635c6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:47:52.675123 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8594274-e392-433f-be44-bec8625635c6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:47:52.677891 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8594274-e392-433f-be44-bec8625635c6 service cinder] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:47:52.721302 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 806/1644] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:47:52 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 10:47:52.834356 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f80cd34-4630-4bd6-bde3-d96d7d2ea577 service cinder] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:47:52.852601 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f80cd34-4630-4bd6-bde3-d96d7d2ea577 service cinder] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:47:52.871244 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8b1012-9376-4745-a508-cc4b49f6980a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '864d35f268cf45148756109759bb2743', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:47:52.871745 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8b1012-9376-4745-a508-cc4b49f6980a service cinder] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:47:52.872025 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8b1012-9376-4745-a508-cc4b49f6980a service cinder] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:47:52.872446 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8b1012-9376-4745-a508-cc4b49f6980a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:47:53.046681 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8b1012-9376-4745-a508-cc4b49f6980a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:47:53.047899 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8b1012-9376-4745-a508-cc4b49f6980a service cinder] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:47:53.078271 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 839/1645] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:47:52 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 244 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 10:47:53.409673 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8594274-e392-433f-be44-bec8625635c6 service cinder] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:47:53.554810 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8594274-e392-433f-be44-bec8625635c6 service cinder] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:47:53.571504 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfafb6a-1419-4b20-a224-2d8609a51d5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:47:53.572178 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfafb6a-1419-4b20-a224-2d8609a51d5f admin admin] REQUEST_METHOD: `DELETE` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:47:53.572459 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfafb6a-1419-4b20-a224-2d8609a51d5f admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:47:53.572729 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfafb6a-1419-4b20-a224-2d8609a51d5f admin admin] PATH_INFO: `/v3/users/6fb6cab163434ef6a1e909e86520c233` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:47:53.574960 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfafb6a-1419-4b20-a224-2d8609a51d5f admin admin] RBAC: Authorizing `identity:delete_user(user_id=6fb6cab163434ef6a1e909e86520c233)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:47:53.577510 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfafb6a-1419-4b20-a224-2d8609a51d5f admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:47:53.697607 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-2dfafb6a-1419-4b20-a224-2d8609a51d5f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6fb6cab163434ef6a1e909e86520c233'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:47:53.707439 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-2dfafb6a-1419-4b20-a224-2d8609a51d5f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6fb6cab163434ef6a1e909e86520c233'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:47:53.727140 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-2dfafb6a-1419-4b20-a224-2d8609a51d5f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6fb6cab163434ef6a1e909e86520c233'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:47:53.727999 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-2dfafb6a-1419-4b20-a224-2d8609a51d5f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6fb6cab163434ef6a1e909e86520c233'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:47:53.728916 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-2dfafb6a-1419-4b20-a224-2d8609a51d5f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6fb6cab163434ef6a1e909e86520c233'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:47:53.759692 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:47:53.779492 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 807/1646] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Jul 27 10:47:53 2022] DELETE /identity/v3/users/6fb6cab163434ef6a1e909e86520c233 => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:47:53.784684 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f7acd8-558a-4fa5-9b8e-fa837a3c9c39 None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:47:53.785709 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f7acd8-558a-4fa5-9b8e-fa837a3c9c39 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:47:53.786310 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f7acd8-558a-4fa5-9b8e-fa837a3c9c39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:47:53.832834 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-73f7acd8-558a-4fa5-9b8e-fa837a3c9c39 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:47:53.835585 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-73f7acd8-558a-4fa5-9b8e-fa837a3c9c39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:47:53.864698 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:47:54.083403 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 840/1647] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:47:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 300 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:47:54.221020 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfafb6a-1419-4b20-a224-2d8609a51d5f admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:47:54.222638 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-2dfafb6a-1419-4b20-a224-2d8609a51d5f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:47:54.423648 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfafb6a-1419-4b20-a224-2d8609a51d5f admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:47:54.450397 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c07d207-bd3e-4735-b432-b26fc5aeb4c3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7922d0a1a7c4e808d0b2a80dca0da40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:47:54.451180 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c07d207-bd3e-4735-b432-b26fc5aeb4c3 service neutron] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:47:54.451689 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c07d207-bd3e-4735-b432-b26fc5aeb4c3 service neutron] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:47:54.452163 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c07d207-bd3e-4735-b432-b26fc5aeb4c3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:47:54.505122 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c07d207-bd3e-4735-b432-b26fc5aeb4c3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:47:54.508850 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c07d207-bd3e-4735-b432-b26fc5aeb4c3 service neutron] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:47:54.543336 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 808/1648] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:47:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 10:47:56.679966 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f7acd8-558a-4fa5-9b8e-fa837a3c9c39 None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:47:56.682810 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-73f7acd8-558a-4fa5-9b8e-fa837a3c9c39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:47:56.705412 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f7acd8-558a-4fa5-9b8e-fa837a3c9c39 None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:47:56.714163 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfe3598-f0af-4b3d-a530-0bd502ea3aa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:47:56.715032 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfe3598-f0af-4b3d-a530-0bd502ea3aa7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:47:56.715479 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfe3598-f0af-4b3d-a530-0bd502ea3aa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:47:56.715963 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfe3598-f0af-4b3d-a530-0bd502ea3aa7 admin admin] PATH_INFO: `/v3/projects/66c9954b29af4eb5a8ca587e290e5c22` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:47:56.730114 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfe3598-f0af-4b3d-a530-0bd502ea3aa7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=66c9954b29af4eb5a8ca587e290e5c22)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:47:56.735586 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfe3598-f0af-4b3d-a530-0bd502ea3aa7 admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:47:56.920178 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-0cfe3598-f0af-4b3d-a530-0bd502ea3aa7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '66c9954b29af4eb5a8ca587e290e5c22', 'request_id': 'req-0cfe3598-f0af-4b3d-a530-0bd502ea3aa7'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:47:56.929345 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-0cfe3598-f0af-4b3d-a530-0bd502ea3aa7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '66c9954b29af4eb5a8ca587e290e5c22', 'request_id': 'req-0cfe3598-f0af-4b3d-a530-0bd502ea3aa7'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:47:56.956048 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-0cfe3598-f0af-4b3d-a530-0bd502ea3aa7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '66c9954b29af4eb5a8ca587e290e5c22', 'request_id': 'req-0cfe3598-f0af-4b3d-a530-0bd502ea3aa7'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:47:56.977406 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-0cfe3598-f0af-4b3d-a530-0bd502ea3aa7 admin admin] The token cache is being invalidate because project 66c9954b29af4eb5a8ca587e290e5c22 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86827) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 10:47:56.978048 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-0cfe3598-f0af-4b3d-a530-0bd502ea3aa7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:47:56.979036 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-0cfe3598-f0af-4b3d-a530-0bd502ea3aa7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:47:56.985276 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 841/1649] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Jul 27 10:47:56 2022] DELETE /identity/v3/projects/66c9954b29af4eb5a8ca587e290e5c22 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 10:47:56.986005 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef4974e-3654-4f0a-9c19-924e1a5d7680 None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:47:56.986471 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef4974e-3654-4f0a-9c19-924e1a5d7680 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:47:56.986545 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef4974e-3654-4f0a-9c19-924e1a5d7680 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:47:57.038916 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-8ef4974e-3654-4f0a-9c19-924e1a5d7680 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:47:57.040644 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-8ef4974e-3654-4f0a-9c19-924e1a5d7680 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:47:57.137149 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d641838-c512-4e71-ac74-c48ab5d8b5cd None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:47:57.137885 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d641838-c512-4e71-ac74-c48ab5d8b5cd None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:47:57.138264 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d641838-c512-4e71-ac74-c48ab5d8b5cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:47:57.186253 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 809/1650] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:47:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:47:57.195715 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56e18f5-52a2-46f3-89f1-5fcaceb8e1d4 None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:47:57.196609 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56e18f5-52a2-46f3-89f1-5fcaceb8e1d4 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:47:57.197680 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56e18f5-52a2-46f3-89f1-5fcaceb8e1d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:47:57.218234 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-6d641838-c512-4e71-ac74-c48ab5d8b5cd None None] MFA Rules not processed for user `798bfb641ab144b28ad904bb8ed6f928`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:47:57.224926 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-6d641838-c512-4e71-ac74-c48ab5d8b5cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:47:57.225902 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-e56e18f5-52a2-46f3-89f1-5fcaceb8e1d4 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:47:57.227424 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-e56e18f5-52a2-46f3-89f1-5fcaceb8e1d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:47:57.244337 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 810/1651] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:47:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:47:57.248865 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56e18f5-52a2-46f3-89f1-5fcaceb8e1d4 None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:47:57.287624 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56e18f5-52a2-46f3-89f1-5fcaceb8e1d4 None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:47:57.301307 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6d75ba-8b44-454b-930e-3459c1798734 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:47:57.302270 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6d75ba-8b44-454b-930e-3459c1798734 admin admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:47:57.302563 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6d75ba-8b44-454b-930e-3459c1798734 admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:47:57.308697 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6d75ba-8b44-454b-930e-3459c1798734 admin admin] PATH_INFO: `/v3/domains` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:47:57.310072 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6d75ba-8b44-454b-930e-3459c1798734 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86828) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:47:57.311780 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6d75ba-8b44-454b-930e-3459c1798734 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:47:57.324486 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6d75ba-8b44-454b-930e-3459c1798734 admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:47:57.362830 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 811/1652] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Jul 27 10:47:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:47:57.366291 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6d75ba-8b44-454b-930e-3459c1798734 admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:47:57.373602 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 842/1653] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:47:57 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 10:47:57.376006 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d641838-c512-4e71-ac74-c48ab5d8b5cd None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:47:57.378173 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-6d641838-c512-4e71-ac74-c48ab5d8b5cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:47:57.404942 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6d75ba-8b44-454b-930e-3459c1798734 admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:47:57.419761 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25004b20-90e0-4d79-b71b-2940d709e92c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:47:57.421574 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25004b20-90e0-4d79-b71b-2940d709e92c admin admin] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:47:57.422676 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25004b20-90e0-4d79-b71b-2940d709e92c admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:47:57.422970 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25004b20-90e0-4d79-b71b-2940d709e92c admin admin] PATH_INFO: `/v3/projects` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:47:57.426053 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25004b20-90e0-4d79-b71b-2940d709e92c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:47:57.426859 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d641838-c512-4e71-ac74-c48ab5d8b5cd None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:47:57.429745 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25004b20-90e0-4d79-b71b-2940d709e92c admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:47:57.443117 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a81ebd4-67e3-45f6-baa5-2211b408b939 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:47:57.445221 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a81ebd4-67e3-45f6-baa5-2211b408b939 admin admin] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:47:57.445515 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a81ebd4-67e3-45f6-baa5-2211b408b939 admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:47:57.446942 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a81ebd4-67e3-45f6-baa5-2211b408b939 admin admin] PATH_INFO: `/v3/projects` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:47:57.448618 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a81ebd4-67e3-45f6-baa5-2211b408b939 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:47:57.451644 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a81ebd4-67e3-45f6-baa5-2211b408b939 admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:47:57.510813 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 812/1654] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Jul 27 10:47:57 2022] POST /identity/v3/projects => generated 373 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 10:47:57.513819 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25004b20-90e0-4d79-b71b-2940d709e92c admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:47:57.537406 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 843/1655] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Jul 27 10:47:57 2022] POST /identity/v3/projects => generated 403 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 10:47:57.539454 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a81ebd4-67e3-45f6-baa5-2211b408b939 admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:47:57.565394 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:47:57.661051 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:DELETE=1,keystone:SELECT=98,keystone:INSERT=1 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:47:57.667570 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25004b20-90e0-4d79-b71b-2940d709e92c admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:47:57.689946 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a81ebd4-67e3-45f6-baa5-2211b408b939 admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:47:57.695179 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20320694-d44a-4a12-83d4-26d8c4318f1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:47:57.696474 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20320694-d44a-4a12-83d4-26d8c4318f1c admin admin] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:47:57.697202 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20320694-d44a-4a12-83d4-26d8c4318f1c admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:47:57.697488 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20320694-d44a-4a12-83d4-26d8c4318f1c admin admin] PATH_INFO: `/v3/users` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:47:57.698174 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20320694-d44a-4a12-83d4-26d8c4318f1c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:47:57.700973 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20320694-d44a-4a12-83d4-26d8c4318f1c admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:47:57.706574 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94116c0c-9d34-4997-bfdf-4fd18145fb71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:47:57.719553 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94116c0c-9d34-4997-bfdf-4fd18145fb71 admin admin] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:47:57.719942 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94116c0c-9d34-4997-bfdf-4fd18145fb71 admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:47:57.720626 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94116c0c-9d34-4997-bfdf-4fd18145fb71 admin admin] PATH_INFO: `/v3/users` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:47:57.723735 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94116c0c-9d34-4997-bfdf-4fd18145fb71 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:47:57.725814 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94116c0c-9d34-4997-bfdf-4fd18145fb71 admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:47:57.734681 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 813/1656] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Jul 27 10:47:57 2022] POST /identity/v3/users => generated 287 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 10:47:57.736814 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20320694-d44a-4a12-83d4-26d8c4318f1c admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:47:57.757658 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20320694-d44a-4a12-83d4-26d8c4318f1c admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:47:57.767254 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6bddbc-80de-4e2b-ac84-0734a77e6cec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:47:57.768749 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6bddbc-80de-4e2b-ac84-0734a77e6cec admin admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:47:57.769300 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6bddbc-80de-4e2b-ac84-0734a77e6cec admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:47:57.769779 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6bddbc-80de-4e2b-ac84-0734a77e6cec admin admin] PATH_INFO: `/v3/roles` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:47:57.771158 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6bddbc-80de-4e2b-ac84-0734a77e6cec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:47:57.775489 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6bddbc-80de-4e2b-ac84-0734a77e6cec admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:47:57.789057 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 844/1657] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Jul 27 10:47:57 2022] POST /identity/v3/users => generated 308 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 10:47:57.795046 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94116c0c-9d34-4997-bfdf-4fd18145fb71 admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:47:57.817547 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 814/1658] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 10:47:57 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 10:47:57.818772 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6bddbc-80de-4e2b-ac84-0734a77e6cec admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:47:57.821079 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94116c0c-9d34-4997-bfdf-4fd18145fb71 admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:47:57.838092 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e268ab99-4104-4f54-ac20-e65873700f45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:47:57.838910 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e268ab99-4104-4f54-ac20-e65873700f45 admin admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:47:57.839299 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e268ab99-4104-4f54-ac20-e65873700f45 admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:47:57.839680 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e268ab99-4104-4f54-ac20-e65873700f45 admin admin] PATH_INFO: `/v3/roles` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:47:57.840495 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e268ab99-4104-4f54-ac20-e65873700f45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:47:57.841523 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6bddbc-80de-4e2b-ac84-0734a77e6cec admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:47:57.842366 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e268ab99-4104-4f54-ac20-e65873700f45 admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:47:57.859415 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f37cbc-8c17-4087-b171-c93c8ff74cda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:47:57.860212 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f37cbc-8c17-4087-b171-c93c8ff74cda admin admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:47:57.860358 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f37cbc-8c17-4087-b171-c93c8ff74cda admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:47:57.860640 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f37cbc-8c17-4087-b171-c93c8ff74cda admin admin] PATH_INFO: `/v3/roles` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:47:57.861359 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f37cbc-8c17-4087-b171-c93c8ff74cda admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:47:57.863097 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f37cbc-8c17-4087-b171-c93c8ff74cda admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:47:57.889146 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 845/1659] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 10:47:57 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 10:47:57.890316 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e268ab99-4104-4f54-ac20-e65873700f45 admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:47:57.890659 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 815/1660] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 10:47:57 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 10:47:57.894517 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f37cbc-8c17-4087-b171-c93c8ff74cda admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:47:57.907971 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e268ab99-4104-4f54-ac20-e65873700f45 admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:47:57.909791 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f37cbc-8c17-4087-b171-c93c8ff74cda admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:47:57.921756 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3068f0c-af24-4ef8-8ea8-22d73d343741 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:47:57.922348 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3068f0c-af24-4ef8-8ea8-22d73d343741 admin admin] REQUEST_METHOD: `PUT` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:47:57.922698 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3068f0c-af24-4ef8-8ea8-22d73d343741 admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:47:57.922963 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3068f0c-af24-4ef8-8ea8-22d73d343741 admin admin] PATH_INFO: `/v3/projects/3ba0e760282a41c9b9cdf72987a4befb/users/2e90ef2394df4855889944a71ef8afb2/roles/4c9fc0b3baed441da2126be41cc6b04a` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:47:57.923207 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9061417b-5c21-4c98-b78a-0b2b49fbce09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:47:57.923900 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9061417b-5c21-4c98-b78a-0b2b49fbce09 admin admin] REQUEST_METHOD: `PUT` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:47:57.924187 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9061417b-5c21-4c98-b78a-0b2b49fbce09 admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:47:57.924466 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9061417b-5c21-4c98-b78a-0b2b49fbce09 admin admin] PATH_INFO: `/v3/projects/727403ef1f3d435d9af07604643eebd1/users/73dd21f6eedd43a492605ad449984d46/roles/21156560eb6b436bb567a40b4976f3f4` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:47:57.981137 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3068f0c-af24-4ef8-8ea8-22d73d343741 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3ba0e760282a41c9b9cdf72987a4befb, user_id=2e90ef2394df4855889944a71ef8afb2, role_id=4c9fc0b3baed441da2126be41cc6b04a)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:47:57.984024 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3068f0c-af24-4ef8-8ea8-22d73d343741 admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:47:57.986166 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9061417b-5c21-4c98-b78a-0b2b49fbce09 admin admin] RBAC: Authorizing `identity:create_grant(project_id=727403ef1f3d435d9af07604643eebd1, user_id=73dd21f6eedd43a492605ad449984d46, role_id=21156560eb6b436bb567a40b4976f3f4)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:47:57.989027 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9061417b-5c21-4c98-b78a-0b2b49fbce09 admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:47:58.025249 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 816/1661] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 10:47:57 2022] PUT /identity/v3/projects/3ba0e760282a41c9b9cdf72987a4befb/users/2e90ef2394df4855889944a71ef8afb2/roles/4c9fc0b3baed441da2126be41cc6b04a => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 10:47:58.032802 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 846/1662] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 10:47:57 2022] PUT /identity/v3/projects/727403ef1f3d435d9af07604643eebd1/users/73dd21f6eedd43a492605ad449984d46/roles/21156560eb6b436bb567a40b4976f3f4 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:47:58.038148 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3068f0c-af24-4ef8-8ea8-22d73d343741 admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:47:58.047899 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38055f18-14a5-40e8-b193-40ccd11838e3 None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:47:58.047899 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38055f18-14a5-40e8-b193-40ccd11838e3 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:47:58.048092 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38055f18-14a5-40e8-b193-40ccd11838e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:47:58.113445 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-38055f18-14a5-40e8-b193-40ccd11838e3 None None] MFA Rules not processed for user `2e90ef2394df4855889944a71ef8afb2`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:47:58.123212 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-38055f18-14a5-40e8-b193-40ccd11838e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:47:58.179846 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3068f0c-af24-4ef8-8ea8-22d73d343741 admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:47:58.273356 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 847/1663] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:47:58 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:47:58.277616 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38055f18-14a5-40e8-b193-40ccd11838e3 None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:47:58.322801 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1023857-fe6a-48ef-a338-96f67d822949 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:47:58.325621 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1023857-fe6a-48ef-a338-96f67d822949 admin admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:47:58.325919 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1023857-fe6a-48ef-a338-96f67d822949 admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:47:58.326265 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1023857-fe6a-48ef-a338-96f67d822949 admin admin] PATH_INFO: `/v3/roles` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:47:58.328638 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1023857-fe6a-48ef-a338-96f67d822949 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:47:58.339995 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1023857-fe6a-48ef-a338-96f67d822949 admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:47:58.380367 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 817/1664] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 10:47:58 2022] GET /identity/v3/roles => generated 1420 bytes in 341 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 10:47:58.381750 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1023857-fe6a-48ef-a338-96f67d822949 admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:47:58.408371 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1023857-fe6a-48ef-a338-96f67d822949 admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:47:58.427083 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6259d9-5231-477f-b97e-dd5634ebed8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:47:58.427775 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6259d9-5231-477f-b97e-dd5634ebed8f admin admin] REQUEST_METHOD: `PUT` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:47:58.428055 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6259d9-5231-477f-b97e-dd5634ebed8f admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:47:58.428323 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6259d9-5231-477f-b97e-dd5634ebed8f admin admin] PATH_INFO: `/v3/projects/727403ef1f3d435d9af07604643eebd1/users/73dd21f6eedd43a492605ad449984d46/roles/21156560eb6b436bb567a40b4976f3f4` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:47:58.432944 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6259d9-5231-477f-b97e-dd5634ebed8f admin admin] RBAC: Authorizing `identity:create_grant(project_id=727403ef1f3d435d9af07604643eebd1, user_id=73dd21f6eedd43a492605ad449984d46, role_id=21156560eb6b436bb567a40b4976f3f4)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:47:58.435577 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:47:58.458286 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6259d9-5231-477f-b97e-dd5634ebed8f admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:47:58.476704 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38055f18-14a5-40e8-b193-40ccd11838e3 None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:47:58.491376 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 818/1665] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 10:47:58 2022] PUT /identity/v3/projects/727403ef1f3d435d9af07604643eebd1/users/73dd21f6eedd43a492605ad449984d46/roles/21156560eb6b436bb567a40b4976f3f4 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:47:58.567638 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb582a41-ee89-462a-a6c4-742a43e1c508 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:47:58.568304 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb582a41-ee89-462a-a6c4-742a43e1c508 admin admin] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:47:58.568590 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb582a41-ee89-462a-a6c4-742a43e1c508 admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:47:58.568869 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb582a41-ee89-462a-a6c4-742a43e1c508 admin admin] PATH_INFO: `/v3/projects` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:47:58.570092 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb582a41-ee89-462a-a6c4-742a43e1c508 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:47:58.572614 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb582a41-ee89-462a-a6c4-742a43e1c508 admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:47:58.624904 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 848/1666] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Jul 27 10:47:58 2022] POST /identity/v3/projects => generated 373 bytes in 348 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 10:47:58.625644 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6259d9-5231-477f-b97e-dd5634ebed8f admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:47:58.639922 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6259d9-5231-477f-b97e-dd5634ebed8f admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:47:58.684008 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:47:58.792736 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c68877-78ee-4057-8f96-f90639568646 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:47:58.793493 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c68877-78ee-4057-8f96-f90639568646 admin admin] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:47:58.794294 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c68877-78ee-4057-8f96-f90639568646 admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:47:58.794644 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c68877-78ee-4057-8f96-f90639568646 admin admin] PATH_INFO: `/v3/users` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:47:58.795329 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c68877-78ee-4057-8f96-f90639568646 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:47:58.798297 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c68877-78ee-4057-8f96-f90639568646 admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:47:58.839151 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb582a41-ee89-462a-a6c4-742a43e1c508 admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:47:58.857259 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 819/1667] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Jul 27 10:47:58 2022] POST /identity/v3/users => generated 293 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 10:47:58.860471 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb582a41-ee89-462a-a6c4-742a43e1c508 admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:47:58.874377 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35216cf-83f5-49fe-81d8-51b8c3845641 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:47:58.875107 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35216cf-83f5-49fe-81d8-51b8c3845641 admin admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:47:58.876113 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35216cf-83f5-49fe-81d8-51b8c3845641 admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:47:58.876399 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35216cf-83f5-49fe-81d8-51b8c3845641 admin admin] PATH_INFO: `/v3/roles` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:47:58.877378 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35216cf-83f5-49fe-81d8-51b8c3845641 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:47:58.879050 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35216cf-83f5-49fe-81d8-51b8c3845641 admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:47:58.907950 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c68877-78ee-4057-8f96-f90639568646 admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:47:58.923819 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 849/1668] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 10:47:58 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 10:47:58.927807 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c68877-78ee-4057-8f96-f90639568646 admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:47:58.939921 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9346955-80d4-4512-aba2-17ef229fe1bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:47:58.940639 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9346955-80d4-4512-aba2-17ef229fe1bb admin admin] REQUEST_METHOD: `PUT` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:47:58.940976 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9346955-80d4-4512-aba2-17ef229fe1bb admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:47:58.941309 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9346955-80d4-4512-aba2-17ef229fe1bb admin admin] PATH_INFO: `/v3/projects/4bc3cd8e5ee24b298f40f77d3e6e765b/users/bf475e55ac124b938b3866875fdb7d30/roles/21156560eb6b436bb567a40b4976f3f4` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:47:58.977700 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9346955-80d4-4512-aba2-17ef229fe1bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=4bc3cd8e5ee24b298f40f77d3e6e765b, user_id=bf475e55ac124b938b3866875fdb7d30, role_id=21156560eb6b436bb567a40b4976f3f4)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:47:58.987759 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9346955-80d4-4512-aba2-17ef229fe1bb admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:47:59.027412 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35216cf-83f5-49fe-81d8-51b8c3845641 admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:47:59.053155 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 820/1669] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 10:47:58 2022] PUT /identity/v3/projects/4bc3cd8e5ee24b298f40f77d3e6e765b/users/bf475e55ac124b938b3866875fdb7d30/roles/21156560eb6b436bb567a40b4976f3f4 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:47:59.073982 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35216cf-83f5-49fe-81d8-51b8c3845641 admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:47:59.266236 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdc8972-571d-4fd1-ba22-c1b9d2f09c49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:47:59.267960 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdc8972-571d-4fd1-ba22-c1b9d2f09c49 admin admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:47:59.269051 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdc8972-571d-4fd1-ba22-c1b9d2f09c49 admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:47:59.269556 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdc8972-571d-4fd1-ba22-c1b9d2f09c49 admin admin] PATH_INFO: `/v3/roles` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:47:59.270627 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bdc8972-571d-4fd1-ba22-c1b9d2f09c49 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:47:59.272578 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bdc8972-571d-4fd1-ba22-c1b9d2f09c49 admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:47:59.290875 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9346955-80d4-4512-aba2-17ef229fe1bb admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:47:59.303604 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9346955-80d4-4512-aba2-17ef229fe1bb admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:47:59.303856 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 850/1670] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 10:47:59 2022] GET /identity/v3/roles => generated 1420 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 10:47:59.317417 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ceab80-fafb-4b06-8a98-05f413072041 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:47:59.318795 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ceab80-fafb-4b06-8a98-05f413072041 admin admin] REQUEST_METHOD: `PUT` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:47:59.318795 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ceab80-fafb-4b06-8a98-05f413072041 admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:47:59.319022 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ceab80-fafb-4b06-8a98-05f413072041 admin admin] PATH_INFO: `/v3/projects/4bc3cd8e5ee24b298f40f77d3e6e765b/users/bf475e55ac124b938b3866875fdb7d30/roles/21156560eb6b436bb567a40b4976f3f4` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:47:59.323385 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ceab80-fafb-4b06-8a98-05f413072041 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4bc3cd8e5ee24b298f40f77d3e6e765b, user_id=bf475e55ac124b938b3866875fdb7d30, role_id=21156560eb6b436bb567a40b4976f3f4)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:47:59.326884 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ceab80-fafb-4b06-8a98-05f413072041 admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:47:59.364074 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 821/1671] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 10:47:59 2022] PUT /identity/v3/projects/4bc3cd8e5ee24b298f40f77d3e6e765b/users/bf475e55ac124b938b3866875fdb7d30/roles/21156560eb6b436bb567a40b4976f3f4 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:47:59.365148 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ec9f6c-861f-499f-8d40-5bdf5a4080d0 None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:47:59.365596 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ec9f6c-861f-499f-8d40-5bdf5a4080d0 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:47:59.366016 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ec9f6c-861f-499f-8d40-5bdf5a4080d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:47:59.437030 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-53ec9f6c-861f-499f-8d40-5bdf5a4080d0 None None] MFA Rules not processed for user `bf475e55ac124b938b3866875fdb7d30`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:47:59.438275 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-53ec9f6c-861f-499f-8d40-5bdf5a4080d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:47:59.589318 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:47:59.615148 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 851/1672] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:47:59 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:47:59.626852 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ceab80-fafb-4b06-8a98-05f413072041 admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:47:59.630924 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-f2ceab80-fafb-4b06-8a98-05f413072041 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:47:59.739867 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ceab80-fafb-4b06-8a98-05f413072041 admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:47:59.745513 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e8491e-bac7-4f3c-8f49-e0ca107cebd4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '864d35f268cf45148756109759bb2743', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:47:59.747737 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e8491e-bac7-4f3c-8f49-e0ca107cebd4 service cinder] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:47:59.747737 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e8491e-bac7-4f3c-8f49-e0ca107cebd4 service cinder] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:47:59.748809 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e8491e-bac7-4f3c-8f49-e0ca107cebd4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:47:59.761861 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e8491e-bac7-4f3c-8f49-e0ca107cebd4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:47:59.763929 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e8491e-bac7-4f3c-8f49-e0ca107cebd4 service cinder] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:47:59.777922 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 822/1673] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:47:59 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 10:47:59.905242 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051dcf0e-b308-4197-8702-91dda6ced04b None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:47:59.906083 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051dcf0e-b308-4197-8702-91dda6ced04b None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:47:59.906375 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051dcf0e-b308-4197-8702-91dda6ced04b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:00.016046 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-051dcf0e-b308-4197-8702-91dda6ced04b None None] MFA Rules not processed for user `5ae9a5696bdc48778835da7a311f20c7`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:48:00.017457 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-051dcf0e-b308-4197-8702-91dda6ced04b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:00.162134 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 852/1674] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:47:59 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:48:00.175488 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b79528-2a49-42c6-9890-2aa7a588e63e None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:00.176177 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b79528-2a49-42c6-9890-2aa7a588e63e None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:00.176177 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b79528-2a49-42c6-9890-2aa7a588e63e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:00.211488 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-68b79528-2a49-42c6-9890-2aa7a588e63e None None] MFA Rules not processed for user `5ae9a5696bdc48778835da7a311f20c7`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:48:00.213702 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-68b79528-2a49-42c6-9890-2aa7a588e63e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:00.249727 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 823/1675] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:48:00 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:48:00.261110 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051dcf0e-b308-4197-8702-91dda6ced04b None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:00.267065 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-051dcf0e-b308-4197-8702-91dda6ced04b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:00.480686 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051dcf0e-b308-4197-8702-91dda6ced04b None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:00.488168 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6fee0d-3d00-4566-ab12-614385110fd6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e059d8b6c28c4fcba925d05d7c98e4ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:00.489004 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6fee0d-3d00-4566-ab12-614385110fd6 service nova] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:00.489510 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6fee0d-3d00-4566-ab12-614385110fd6 service nova] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:00.490515 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6fee0d-3d00-4566-ab12-614385110fd6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:00.500846 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6fee0d-3d00-4566-ab12-614385110fd6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:00.508781 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6fee0d-3d00-4566-ab12-614385110fd6 service nova] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:00.520052 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 853/1676] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:48:00 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 259 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 10:48:00.661802 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b79528-2a49-42c6-9890-2aa7a588e63e None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:00.802917 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b79528-2a49-42c6-9890-2aa7a588e63e None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:00.816335 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebc8683-f178-4d9e-a016-8bd42b173df3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:00.817016 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebc8683-f178-4d9e-a016-8bd42b173df3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:00.817303 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebc8683-f178-4d9e-a016-8bd42b173df3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:00.817587 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebc8683-f178-4d9e-a016-8bd42b173df3 admin admin] PATH_INFO: `/v3/users/2e90ef2394df4855889944a71ef8afb2` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:00.818857 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebc8683-f178-4d9e-a016-8bd42b173df3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2e90ef2394df4855889944a71ef8afb2)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:00.820832 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebc8683-f178-4d9e-a016-8bd42b173df3 admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:00.879873 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=93,keystone:INSERT=5,keystone:DELETE=2 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:48:00.954123 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-eebc8683-f178-4d9e-a016-8bd42b173df3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2e90ef2394df4855889944a71ef8afb2'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:00.977068 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-eebc8683-f178-4d9e-a016-8bd42b173df3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2e90ef2394df4855889944a71ef8afb2'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:01.000413 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb5cd7f-e7b0-4d8d-a726-0561fb55d83e None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:01.000855 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb5cd7f-e7b0-4d8d-a726-0561fb55d83e None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:01.000961 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb5cd7f-e7b0-4d8d-a726-0561fb55d83e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:01.008095 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-eebc8683-f178-4d9e-a016-8bd42b173df3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2e90ef2394df4855889944a71ef8afb2'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:01.008831 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-eebc8683-f178-4d9e-a016-8bd42b173df3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2e90ef2394df4855889944a71ef8afb2'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:01.009440 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-eebc8683-f178-4d9e-a016-8bd42b173df3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2e90ef2394df4855889944a71ef8afb2'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:01.053096 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 824/1677] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Jul 27 10:48:00 2022] DELETE /identity/v3/users/2e90ef2394df4855889944a71ef8afb2 => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:48:01.057136 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f6567c-9823-489e-8a8e-2fd2ed0d9dd0 None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:01.057699 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f6567c-9823-489e-8a8e-2fd2ed0d9dd0 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:01.059094 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f6567c-9823-489e-8a8e-2fd2ed0d9dd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:01.061647 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-3eb5cd7f-e7b0-4d8d-a726-0561fb55d83e None None] MFA Rules not processed for user `73dd21f6eedd43a492605ad449984d46`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:48:01.063577 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-3eb5cd7f-e7b0-4d8d-a726-0561fb55d83e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:01.118520 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-23f6567c-9823-489e-8a8e-2fd2ed0d9dd0 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:48:01.120665 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-23f6567c-9823-489e-8a8e-2fd2ed0d9dd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:01.239055 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:48:01.264665 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 854/1678] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:48:00 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 10:48:01.266778 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 825/1679] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:48:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 10:48:01.386535 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb5cd7f-e7b0-4d8d-a726-0561fb55d83e None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:01.388455 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-3eb5cd7f-e7b0-4d8d-a726-0561fb55d83e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:01.523441 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb5cd7f-e7b0-4d8d-a726-0561fb55d83e None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:01.526794 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ea6e51-e963-42ee-a4c9-3214eafadcba None None] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:01.527888 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ea6e51-e963-42ee-a4c9-3214eafadcba None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:01.529287 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ea6e51-e963-42ee-a4c9-3214eafadcba None None] PATH_INFO: `/` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:01.531175 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 826/1680] 10.222.0.36 () {54 vars in 912 bytes} [Wed Jul 27 10:48:01 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 10:48:01.534867 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e60d5f-648d-49f2-9298-ec39ff3ebf8a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7922d0a1a7c4e808d0b2a80dca0da40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:01.538070 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e60d5f-648d-49f2-9298-ec39ff3ebf8a service neutron] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:01.538070 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e60d5f-648d-49f2-9298-ec39ff3ebf8a service neutron] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:01.538367 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e60d5f-648d-49f2-9298-ec39ff3ebf8a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:01.556786 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7743fa2a-a28b-434f-afe1-0932d0a49e28 None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:01.557386 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7743fa2a-a28b-434f-afe1-0932d0a49e28 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:01.557890 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7743fa2a-a28b-434f-afe1-0932d0a49e28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:01.565616 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e60d5f-648d-49f2-9298-ec39ff3ebf8a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:01.570465 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e60d5f-648d-49f2-9298-ec39ff3ebf8a service neutron] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:01.608581 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 855/1681] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:48:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 10:48:01.635014 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334da09d-5bb3-4a74-87f8-9ee1fc5fdce6 None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:01.636947 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334da09d-5bb3-4a74-87f8-9ee1fc5fdce6 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:01.637397 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334da09d-5bb3-4a74-87f8-9ee1fc5fdce6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:01.662136 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-7743fa2a-a28b-434f-afe1-0932d0a49e28 None None] MFA Rules not processed for user `c7922d0a1a7c4e808d0b2a80dca0da40`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:48:01.663235 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-7743fa2a-a28b-434f-afe1-0932d0a49e28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:01.686766 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 827/1682] 10.222.0.36 () {58 vars in 1009 bytes} [Wed Jul 27 10:48:01 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 10:48:01.719652 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-334da09d-5bb3-4a74-87f8-9ee1fc5fdce6 None None] MFA Rules not processed for user `9b094c00959246f5a113f600fb07215d`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:48:01.720908 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-334da09d-5bb3-4a74-87f8-9ee1fc5fdce6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:01.786230 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7743fa2a-a28b-434f-afe1-0932d0a49e28 None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:01.798184 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7743fa2a-a28b-434f-afe1-0932d0a49e28 None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:01.818145 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591c6b90-8a92-4679-9ab9-a226996e5662 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7922d0a1a7c4e808d0b2a80dca0da40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:01.819112 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591c6b90-8a92-4679-9ab9-a226996e5662 service neutron] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:01.819509 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591c6b90-8a92-4679-9ab9-a226996e5662 service neutron] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:01.819883 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591c6b90-8a92-4679-9ab9-a226996e5662 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:01.843611 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591c6b90-8a92-4679-9ab9-a226996e5662 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:01.846546 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591c6b90-8a92-4679-9ab9-a226996e5662 service neutron] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:01.862924 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 828/1683] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:48:01 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 76 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 10:48:01.910747 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 856/1684] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:48:01 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 272 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:48:01.920999 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591c6b90-8a92-4679-9ab9-a226996e5662 service neutron] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:01.922475 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-591c6b90-8a92-4679-9ab9-a226996e5662 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:02.088337 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:48:02.101269 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591c6b90-8a92-4679-9ab9-a226996e5662 service neutron] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:02.119596 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9229bfd8-467c-4a63-a7d0-3c7174285722 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e059d8b6c28c4fcba925d05d7c98e4ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:02.120254 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9229bfd8-467c-4a63-a7d0-3c7174285722 service nova] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:02.120540 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9229bfd8-467c-4a63-a7d0-3c7174285722 service nova] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:02.120820 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9229bfd8-467c-4a63-a7d0-3c7174285722 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:02.141306 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9229bfd8-467c-4a63-a7d0-3c7174285722 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:02.143320 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9229bfd8-467c-4a63-a7d0-3c7174285722 service nova] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:02.168513 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 829/1685] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:48:01 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 248 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 10:48:02.750821 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334da09d-5bb3-4a74-87f8-9ee1fc5fdce6 None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:02.753023 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-334da09d-5bb3-4a74-87f8-9ee1fc5fdce6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:02.779310 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334da09d-5bb3-4a74-87f8-9ee1fc5fdce6 None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:02.798297 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fecbf52-6549-4ce0-baaf-c1b5c936d12e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:02.800367 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fecbf52-6549-4ce0-baaf-c1b5c936d12e admin admin] REQUEST_METHOD: `DELETE` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:02.803199 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fecbf52-6549-4ce0-baaf-c1b5c936d12e admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:02.806181 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fecbf52-6549-4ce0-baaf-c1b5c936d12e admin admin] PATH_INFO: `/v3/projects/3ba0e760282a41c9b9cdf72987a4befb` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:02.813255 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fecbf52-6549-4ce0-baaf-c1b5c936d12e admin admin] RBAC: Authorizing `identity:delete_project(project_id=3ba0e760282a41c9b9cdf72987a4befb)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:02.815378 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fecbf52-6549-4ce0-baaf-c1b5c936d12e admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:02.998813 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-4fecbf52-6549-4ce0-baaf-c1b5c936d12e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3ba0e760282a41c9b9cdf72987a4befb', 'request_id': 'req-4fecbf52-6549-4ce0-baaf-c1b5c936d12e'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:03.009737 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-4fecbf52-6549-4ce0-baaf-c1b5c936d12e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3ba0e760282a41c9b9cdf72987a4befb', 'request_id': 'req-4fecbf52-6549-4ce0-baaf-c1b5c936d12e'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:03.033821 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-4fecbf52-6549-4ce0-baaf-c1b5c936d12e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3ba0e760282a41c9b9cdf72987a4befb', 'request_id': 'req-4fecbf52-6549-4ce0-baaf-c1b5c936d12e'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:03.053287 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-4fecbf52-6549-4ce0-baaf-c1b5c936d12e admin admin] The token cache is being invalidate because project 3ba0e760282a41c9b9cdf72987a4befb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86827) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 10:48:03.053767 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-4fecbf52-6549-4ce0-baaf-c1b5c936d12e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:03.054591 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-4fecbf52-6549-4ce0-baaf-c1b5c936d12e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:03.056603 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 857/1686] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Jul 27 10:48:02 2022] DELETE /identity/v3/projects/3ba0e760282a41c9b9cdf72987a4befb => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:48:03.060904 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9229bfd8-467c-4a63-a7d0-3c7174285722 service nova] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:03.095549 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9229bfd8-467c-4a63-a7d0-3c7174285722 service nova] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:03.258100 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6cbdc6-0d57-42a2-9575-ce66a36ed7af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:03.258788 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6cbdc6-0d57-42a2-9575-ce66a36ed7af admin admin] REQUEST_METHOD: `DELETE` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:03.259255 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6cbdc6-0d57-42a2-9575-ce66a36ed7af admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:03.259681 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6cbdc6-0d57-42a2-9575-ce66a36ed7af admin admin] PATH_INFO: `/v3/users/bf475e55ac124b938b3866875fdb7d30` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:03.261062 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6cbdc6-0d57-42a2-9575-ce66a36ed7af admin admin] RBAC: Authorizing `identity:delete_user(user_id=bf475e55ac124b938b3866875fdb7d30)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:03.268424 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6cbdc6-0d57-42a2-9575-ce66a36ed7af admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:03.376802 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0b8c33-ae2b-4546-8ce3-a44d6db08869 None None] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:03.377330 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0b8c33-ae2b-4546-8ce3-a44d6db08869 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:03.377603 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0b8c33-ae2b-4546-8ce3-a44d6db08869 None None] PATH_INFO: `/` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:03.379119 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 858/1687] 10.222.0.36 () {54 vars in 921 bytes} [Wed Jul 27 10:48:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 10:48:03.386619 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-0a6cbdc6-0d57-42a2-9575-ce66a36ed7af admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bf475e55ac124b938b3866875fdb7d30'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:03.406773 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-0a6cbdc6-0d57-42a2-9575-ce66a36ed7af admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bf475e55ac124b938b3866875fdb7d30'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:03.429883 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-0a6cbdc6-0d57-42a2-9575-ce66a36ed7af admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bf475e55ac124b938b3866875fdb7d30'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:03.430826 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-0a6cbdc6-0d57-42a2-9575-ce66a36ed7af admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bf475e55ac124b938b3866875fdb7d30'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:03.433416 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-0a6cbdc6-0d57-42a2-9575-ce66a36ed7af admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bf475e55ac124b938b3866875fdb7d30'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:03.440139 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615fa0e0-b601-4628-aae0-c4bbff18df25 None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:03.440466 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615fa0e0-b601-4628-aae0-c4bbff18df25 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:03.441924 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615fa0e0-b601-4628-aae0-c4bbff18df25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:03.472158 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 830/1688] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Jul 27 10:48:03 2022] DELETE /identity/v3/users/bf475e55ac124b938b3866875fdb7d30 => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:48:03.509779 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-615fa0e0-b601-4628-aae0-c4bbff18df25 None None] MFA Rules not processed for user `16cc6179e9a04b6398fbacb40ad1c555`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:48:03.511045 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-615fa0e0-b601-4628-aae0-c4bbff18df25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:03.540764 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:48:03.694698 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67ed0eb-d7d3-40e0-9e2f-be272bb7e0b0 None None] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:03.694987 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67ed0eb-d7d3-40e0-9e2f-be272bb7e0b0 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:03.695496 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67ed0eb-d7d3-40e0-9e2f-be272bb7e0b0 None None] PATH_INFO: `/` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:03.697155 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 831/1689] 10.222.0.36 () {54 vars in 961 bytes} [Wed Jul 27 10:48:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 10:48:03.714237 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0542c1-959e-471c-8552-07ca1d1a0dc6 None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:03.715033 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0542c1-959e-471c-8552-07ca1d1a0dc6 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:03.716140 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0542c1-959e-471c-8552-07ca1d1a0dc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:03.730114 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 859/1690] 10.222.0.36 () {58 vars in 1018 bytes} [Wed Jul 27 10:48:03 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 291 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 10:48:03.775032 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-eb0542c1-959e-471c-8552-07ca1d1a0dc6 None None] MFA Rules not processed for user `16cc6179e9a04b6398fbacb40ad1c555`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:48:03.783627 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-eb0542c1-959e-471c-8552-07ca1d1a0dc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:03.826321 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615fa0e0-b601-4628-aae0-c4bbff18df25 None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:03.826907 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 832/1691] 10.222.0.36 () {58 vars in 1058 bytes} [Wed Jul 27 10:48:03 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (3 switches on core 0) Jul 27 10:48:03.847142 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615fa0e0-b601-4628-aae0-c4bbff18df25 None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:03.863365 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f294d6-13e8-4c77-9f43-c444ea2ba3ad service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16cc6179e9a04b6398fbacb40ad1c555', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:03.864021 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f294d6-13e8-4c77-9f43-c444ea2ba3ad service placement] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:03.864478 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f294d6-13e8-4c77-9f43-c444ea2ba3ad service placement] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:03.864746 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f294d6-13e8-4c77-9f43-c444ea2ba3ad service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:03.872689 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-60f294d6-13e8-4c77-9f43-c444ea2ba3ad service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:03.903380 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f294d6-13e8-4c77-9f43-c444ea2ba3ad service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:03.913646 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f294d6-13e8-4c77-9f43-c444ea2ba3ad service placement] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:03.949287 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 860/1692] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:48:03 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 10:48:04.106392 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0542c1-959e-471c-8552-07ca1d1a0dc6 None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:04.108046 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-eb0542c1-959e-471c-8552-07ca1d1a0dc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:04.143370 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0542c1-959e-471c-8552-07ca1d1a0dc6 None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:04.171514 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ce79fc-67b3-4fdb-ad12-240b23f9db18 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16cc6179e9a04b6398fbacb40ad1c555', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:04.172524 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ce79fc-67b3-4fdb-ad12-240b23f9db18 service placement] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:04.174123 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ce79fc-67b3-4fdb-ad12-240b23f9db18 service placement] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:04.174859 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ce79fc-67b3-4fdb-ad12-240b23f9db18 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:04.205560 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ce79fc-67b3-4fdb-ad12-240b23f9db18 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:04.219737 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ce79fc-67b3-4fdb-ad12-240b23f9db18 service placement] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:04.255130 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 833/1693] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:48:04 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 10:48:04.433323 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb287bce-d009-48ce-bba8-2381ff2509bd None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:04.436562 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb287bce-d009-48ce-bba8-2381ff2509bd None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:04.437043 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb287bce-d009-48ce-bba8-2381ff2509bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:04.496048 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-fb287bce-d009-48ce-bba8-2381ff2509bd None None] MFA Rules not processed for user `5ae9a5696bdc48778835da7a311f20c7`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:48:04.524941 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-fb287bce-d009-48ce-bba8-2381ff2509bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:04.727253 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 861/1694] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:48:04 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 296 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:48:04.845787 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ce79fc-67b3-4fdb-ad12-240b23f9db18 service placement] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:04.847349 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-93ce79fc-67b3-4fdb-ad12-240b23f9db18 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:05.021130 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:48:05.071232 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ce79fc-67b3-4fdb-ad12-240b23f9db18 service placement] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:05.084714 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28beccf9-d1a0-4521-ab41-ed0474e5ca01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:05.085492 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28beccf9-d1a0-4521-ab41-ed0474e5ca01 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:05.092591 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28beccf9-d1a0-4521-ab41-ed0474e5ca01 admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:05.099202 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28beccf9-d1a0-4521-ab41-ed0474e5ca01 admin admin] PATH_INFO: `/v3/projects/4bc3cd8e5ee24b298f40f77d3e6e765b` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:05.101991 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28beccf9-d1a0-4521-ab41-ed0474e5ca01 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4bc3cd8e5ee24b298f40f77d3e6e765b)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:05.103172 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: WARNING py.warnings [None req-28beccf9-d1a0-4521-ab41-ed0474e5ca01 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 27 10:48:05.103172 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: warnings.warn(msg) Jul 27 10:48:05.103172 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: Jul 27 10:48:05.105598 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28beccf9-d1a0-4521-ab41-ed0474e5ca01 admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:05.206976 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3784433c-409b-4598-a889-4819b20af3aa None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:05.207932 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3784433c-409b-4598-a889-4819b20af3aa None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:05.208395 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3784433c-409b-4598-a889-4819b20af3aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:05.267775 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-28beccf9-d1a0-4521-ab41-ed0474e5ca01 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4bc3cd8e5ee24b298f40f77d3e6e765b', 'request_id': 'req-28beccf9-d1a0-4521-ab41-ed0474e5ca01'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:05.296141 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-28beccf9-d1a0-4521-ab41-ed0474e5ca01 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4bc3cd8e5ee24b298f40f77d3e6e765b', 'request_id': 'req-28beccf9-d1a0-4521-ab41-ed0474e5ca01'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:05.299238 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-3784433c-409b-4598-a889-4819b20af3aa None None] MFA Rules not processed for user `b82963d0f8f245aaa295379a38908c4d`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:48:05.300492 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-3784433c-409b-4598-a889-4819b20af3aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:05.329671 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-28beccf9-d1a0-4521-ab41-ed0474e5ca01 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4bc3cd8e5ee24b298f40f77d3e6e765b', 'request_id': 'req-28beccf9-d1a0-4521-ab41-ed0474e5ca01'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:05.373237 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-28beccf9-d1a0-4521-ab41-ed0474e5ca01 admin admin] The token cache is being invalidate because project 4bc3cd8e5ee24b298f40f77d3e6e765b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86828) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 10:48:05.373851 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-28beccf9-d1a0-4521-ab41-ed0474e5ca01 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:05.375458 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-28beccf9-d1a0-4521-ab41-ed0474e5ca01 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:05.377559 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 834/1695] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Jul 27 10:48:04 2022] DELETE /identity/v3/projects/4bc3cd8e5ee24b298f40f77d3e6e765b => generated 0 bytes in 532 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:48:05.378765 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28beccf9-d1a0-4521-ab41-ed0474e5ca01 admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:05.380565 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-28beccf9-d1a0-4521-ab41-ed0474e5ca01 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:05.508413 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 862/1696] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:48:05 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 303 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:48:05.527601 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3710381-26a5-4519-88cb-066918705f61 None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:05.528212 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3710381-26a5-4519-88cb-066918705f61 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:05.530710 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3710381-26a5-4519-88cb-066918705f61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:05.620717 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-e3710381-26a5-4519-88cb-066918705f61 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:48:05.635340 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-e3710381-26a5-4519-88cb-066918705f61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:05.661290 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28beccf9-d1a0-4521-ab41-ed0474e5ca01 admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:05.679693 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 863/1697] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:48:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:48:05.685029 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b94d8b-bf21-4d22-bead-fc7e0e2124f2 None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:05.689518 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b94d8b-bf21-4d22-bead-fc7e0e2124f2 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:05.690573 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b94d8b-bf21-4d22-bead-fc7e0e2124f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:05.699640 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03ce0ad-8425-4c72-9576-300a6dc1a83f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:05.700745 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03ce0ad-8425-4c72-9576-300a6dc1a83f admin admin] REQUEST_METHOD: `DELETE` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:05.701256 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03ce0ad-8425-4c72-9576-300a6dc1a83f admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:05.701743 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03ce0ad-8425-4c72-9576-300a6dc1a83f admin admin] PATH_INFO: `/v3/users/5add2ee95b6c47a783898368f1bf66d0` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:05.703866 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03ce0ad-8425-4c72-9576-300a6dc1a83f admin admin] RBAC: Authorizing `identity:delete_user(user_id=5add2ee95b6c47a783898368f1bf66d0)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:05.719535 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03ce0ad-8425-4c72-9576-300a6dc1a83f admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:05.790767 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-52b94d8b-bf21-4d22-bead-fc7e0e2124f2 None None] MFA Rules not processed for user `b82963d0f8f245aaa295379a38908c4d`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:48:05.792331 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-52b94d8b-bf21-4d22-bead-fc7e0e2124f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:05.821499 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 864/1698] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:48:05 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:48:05.825377 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b313a42-5522-4e00-8901-4e28bb1d950c None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:05.826321 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b313a42-5522-4e00-8901-4e28bb1d950c None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:05.826743 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b313a42-5522-4e00-8901-4e28bb1d950c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:05.866364 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-c03ce0ad-8425-4c72-9576-300a6dc1a83f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5add2ee95b6c47a783898368f1bf66d0'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:05.878791 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-2b313a42-5522-4e00-8901-4e28bb1d950c None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:48:05.881350 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-2b313a42-5522-4e00-8901-4e28bb1d950c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:05.882437 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-c03ce0ad-8425-4c72-9576-300a6dc1a83f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5add2ee95b6c47a783898368f1bf66d0'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:05.898929 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-c03ce0ad-8425-4c72-9576-300a6dc1a83f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5add2ee95b6c47a783898368f1bf66d0'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:05.899776 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-c03ce0ad-8425-4c72-9576-300a6dc1a83f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5add2ee95b6c47a783898368f1bf66d0'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:05.900513 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-c03ce0ad-8425-4c72-9576-300a6dc1a83f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5add2ee95b6c47a783898368f1bf66d0'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:05.906564 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 865/1699] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:48:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:48:05.909937 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b313a42-5522-4e00-8901-4e28bb1d950c None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:05.909937 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-2b313a42-5522-4e00-8901-4e28bb1d950c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:05.927931 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:48:05.931005 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:48:05.942107 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 835/1700] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Jul 27 10:48:05 2022] DELETE /identity/v3/users/5add2ee95b6c47a783898368f1bf66d0 => generated 0 bytes in 562 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:48:05.943498 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03ce0ad-8425-4c72-9576-300a6dc1a83f admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:05.946864 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-c03ce0ad-8425-4c72-9576-300a6dc1a83f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:06.080261 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b313a42-5522-4e00-8901-4e28bb1d950c None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:06.124182 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03ce0ad-8425-4c72-9576-300a6dc1a83f admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:06.145279 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d7eb1c-0549-434f-b748-4da27b3ff223 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:06.146341 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d7eb1c-0549-434f-b748-4da27b3ff223 admin admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:06.147160 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d7eb1c-0549-434f-b748-4da27b3ff223 admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:06.148127 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d7eb1c-0549-434f-b748-4da27b3ff223 admin admin] PATH_INFO: `/v3/domains` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:06.151226 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d7eb1c-0549-434f-b748-4da27b3ff223 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86828) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:48:06.153280 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d7eb1c-0549-434f-b748-4da27b3ff223 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:06.156824 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d7eb1c-0549-434f-b748-4da27b3ff223 admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:06.183002 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 836/1701] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Jul 27 10:48:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:48:06.187456 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5cc2f6-4c5d-4355-b97e-54da6db6d20e None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:06.188185 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5cc2f6-4c5d-4355-b97e-54da6db6d20e None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:06.188703 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5cc2f6-4c5d-4355-b97e-54da6db6d20e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:06.242761 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-5a5cc2f6-4c5d-4355-b97e-54da6db6d20e None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:48:06.245085 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-5a5cc2f6-4c5d-4355-b97e-54da6db6d20e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:06.264476 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707013ae-88b6-4206-a90d-3042f789ccfd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e059d8b6c28c4fcba925d05d7c98e4ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:06.270218 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707013ae-88b6-4206-a90d-3042f789ccfd service nova] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:06.270768 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707013ae-88b6-4206-a90d-3042f789ccfd service nova] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:06.271374 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707013ae-88b6-4206-a90d-3042f789ccfd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:06.276408 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 837/1702] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:48:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:48:06.277759 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-707013ae-88b6-4206-a90d-3042f789ccfd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:06.279586 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5cc2f6-4c5d-4355-b97e-54da6db6d20e None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:06.303962 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5cc2f6-4c5d-4355-b97e-54da6db6d20e None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:06.318164 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d699c73-9fd7-4062-b4db-07cbfd9d1c33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:06.319103 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d699c73-9fd7-4062-b4db-07cbfd9d1c33 admin admin] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:06.319586 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d699c73-9fd7-4062-b4db-07cbfd9d1c33 admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:06.320278 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d699c73-9fd7-4062-b4db-07cbfd9d1c33 admin admin] PATH_INFO: `/v3/projects` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:06.321367 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d699c73-9fd7-4062-b4db-07cbfd9d1c33 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:06.327418 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d699c73-9fd7-4062-b4db-07cbfd9d1c33 admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:06.391572 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 838/1703] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Jul 27 10:48:06 2022] POST /identity/v3/projects => generated 369 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 10:48:06.395476 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d699c73-9fd7-4062-b4db-07cbfd9d1c33 admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:06.465215 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707013ae-88b6-4206-a90d-3042f789ccfd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:06.473444 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707013ae-88b6-4206-a90d-3042f789ccfd service nova] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:06.506498 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 866/1704] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:48:05 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 599 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 10:48:06.617249 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707013ae-88b6-4206-a90d-3042f789ccfd service nova] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:06.619552 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-707013ae-88b6-4206-a90d-3042f789ccfd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:06.657292 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d699c73-9fd7-4062-b4db-07cbfd9d1c33 admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:06.674818 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec46ae3-1d0f-439e-86e4-85f9748cd979 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:06.676589 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec46ae3-1d0f-439e-86e4-85f9748cd979 admin admin] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:06.678262 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec46ae3-1d0f-439e-86e4-85f9748cd979 admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:06.679491 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec46ae3-1d0f-439e-86e4-85f9748cd979 admin admin] PATH_INFO: `/v3/users` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:06.682710 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec46ae3-1d0f-439e-86e4-85f9748cd979 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:06.703880 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec46ae3-1d0f-439e-86e4-85f9748cd979 admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:06.762828 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 839/1705] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Jul 27 10:48:06 2022] POST /identity/v3/users => generated 285 bytes in 368 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 10:48:06.766562 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec46ae3-1d0f-439e-86e4-85f9748cd979 admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:06.776464 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:48:06.785087 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec46ae3-1d0f-439e-86e4-85f9748cd979 admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:06.801316 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3716f92-38c3-4fb0-8a95-4863c478bc34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:06.801973 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3716f92-38c3-4fb0-8a95-4863c478bc34 admin admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:06.802260 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3716f92-38c3-4fb0-8a95-4863c478bc34 admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:06.802651 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3716f92-38c3-4fb0-8a95-4863c478bc34 admin admin] PATH_INFO: `/v3/roles` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:06.803844 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707013ae-88b6-4206-a90d-3042f789ccfd service nova] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:06.811511 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3716f92-38c3-4fb0-8a95-4863c478bc34 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:06.813171 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3716f92-38c3-4fb0-8a95-4863c478bc34 admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:06.846456 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 840/1706] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 10:48:06 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 10:48:06.852576 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3716f92-38c3-4fb0-8a95-4863c478bc34 admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:06.876710 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3716f92-38c3-4fb0-8a95-4863c478bc34 admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:06.909981 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c1257c-7660-477d-ba50-9f9b1bfd60eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:06.911865 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c1257c-7660-477d-ba50-9f9b1bfd60eb admin admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:06.913406 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c1257c-7660-477d-ba50-9f9b1bfd60eb admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:06.914213 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c1257c-7660-477d-ba50-9f9b1bfd60eb admin admin] PATH_INFO: `/v3/roles` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:06.916235 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c1257c-7660-477d-ba50-9f9b1bfd60eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:06.920301 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c1257c-7660-477d-ba50-9f9b1bfd60eb admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:06.962905 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 841/1707] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 10:48:06 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 10:48:06.964988 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c1257c-7660-477d-ba50-9f9b1bfd60eb admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:06.969780 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-81c1257c-7660-477d-ba50-9f9b1bfd60eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:06.980128 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0d42fe-2c44-4333-86d7-e7d46836970f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7922d0a1a7c4e808d0b2a80dca0da40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:06.980924 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0d42fe-2c44-4333-86d7-e7d46836970f service neutron] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:06.981244 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0d42fe-2c44-4333-86d7-e7d46836970f service neutron] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:06.981523 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0d42fe-2c44-4333-86d7-e7d46836970f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:06.999960 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0d42fe-2c44-4333-86d7-e7d46836970f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:07.001781 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0d42fe-2c44-4333-86d7-e7d46836970f service neutron] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:07.006926 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c1257c-7660-477d-ba50-9f9b1bfd60eb admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:07.020793 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 867/1708] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:48:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 10:48:07.022000 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0d42fe-2c44-4333-86d7-e7d46836970f service neutron] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:07.027624 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d4ef4c-0332-4756-99f4-56965f5fe3c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:07.029094 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d4ef4c-0332-4756-99f4-56965f5fe3c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:07.029519 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d4ef4c-0332-4756-99f4-56965f5fe3c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:07.029923 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d4ef4c-0332-4756-99f4-56965f5fe3c5 admin admin] PATH_INFO: `/v3/users/798bfb641ab144b28ad904bb8ed6f928` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:07.032735 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d4ef4c-0332-4756-99f4-56965f5fe3c5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=798bfb641ab144b28ad904bb8ed6f928)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:07.037138 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0d42fe-2c44-4333-86d7-e7d46836970f service neutron] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:07.038227 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d4ef4c-0332-4756-99f4-56965f5fe3c5 admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:07.047260 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0534bd7d-bf12-4276-8603-7f1bce42a04b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:07.048165 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0534bd7d-bf12-4276-8603-7f1bce42a04b admin admin] REQUEST_METHOD: `PUT` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:07.048684 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0534bd7d-bf12-4276-8603-7f1bce42a04b admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:07.049115 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0534bd7d-bf12-4276-8603-7f1bce42a04b admin admin] PATH_INFO: `/v3/projects/41bc095eb14146d9b1bcbc6b6e12a28e/users/0e30b5c2011e40f2aa680f0dee4512a2/roles/4c9fc0b3baed441da2126be41cc6b04a` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:07.067984 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0534bd7d-bf12-4276-8603-7f1bce42a04b admin admin] RBAC: Authorizing `identity:create_grant(project_id=41bc095eb14146d9b1bcbc6b6e12a28e, user_id=0e30b5c2011e40f2aa680f0dee4512a2, role_id=4c9fc0b3baed441da2126be41cc6b04a)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:07.070783 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0534bd7d-bf12-4276-8603-7f1bce42a04b admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:07.115205 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 868/1709] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 10:48:07 2022] PUT /identity/v3/projects/41bc095eb14146d9b1bcbc6b6e12a28e/users/0e30b5c2011e40f2aa680f0dee4512a2/roles/4c9fc0b3baed441da2126be41cc6b04a => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:48:07.118554 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507f5003-e406-431d-96d5-de7d792d17a9 None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:07.118787 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507f5003-e406-431d-96d5-de7d792d17a9 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:07.119554 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507f5003-e406-431d-96d5-de7d792d17a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:07.129033 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=92,keystone:INSERT=4,keystone:DELETE=4 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:48:07.159745 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-e7d4ef4c-0332-4756-99f4-56965f5fe3c5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '798bfb641ab144b28ad904bb8ed6f928'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:07.168757 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-507f5003-e406-431d-96d5-de7d792d17a9 None None] MFA Rules not processed for user `0e30b5c2011e40f2aa680f0dee4512a2`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:48:07.170991 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-507f5003-e406-431d-96d5-de7d792d17a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:07.180094 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-e7d4ef4c-0332-4756-99f4-56965f5fe3c5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '798bfb641ab144b28ad904bb8ed6f928'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:07.228901 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-e7d4ef4c-0332-4756-99f4-56965f5fe3c5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '798bfb641ab144b28ad904bb8ed6f928'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:07.229704 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-e7d4ef4c-0332-4756-99f4-56965f5fe3c5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '798bfb641ab144b28ad904bb8ed6f928'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:07.233200 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-e7d4ef4c-0332-4756-99f4-56965f5fe3c5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '798bfb641ab144b28ad904bb8ed6f928'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:07.272022 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 842/1710] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Jul 27 10:48:06 2022] DELETE /identity/v3/users/798bfb641ab144b28ad904bb8ed6f928 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:48:07.282323 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 869/1711] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:48:07 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:48:07.287841 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d4ef4c-0332-4756-99f4-56965f5fe3c5 admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:07.289337 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-e7d4ef4c-0332-4756-99f4-56965f5fe3c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:07.395102 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d4ef4c-0332-4756-99f4-56965f5fe3c5 admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:07.406309 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4776587e-c975-479f-8ad4-f92632655171 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:07.407366 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4776587e-c975-479f-8ad4-f92632655171 admin admin] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:07.407366 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4776587e-c975-479f-8ad4-f92632655171 admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:07.407559 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4776587e-c975-479f-8ad4-f92632655171 admin admin] PATH_INFO: `/v3/projects` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:07.408201 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4776587e-c975-479f-8ad4-f92632655171 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:07.410171 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4776587e-c975-479f-8ad4-f92632655171 admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:07.459561 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9120bbff-5927-4f1a-b276-883665fc0aa9 None None] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:07.459881 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9120bbff-5927-4f1a-b276-883665fc0aa9 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:07.460213 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9120bbff-5927-4f1a-b276-883665fc0aa9 None None] PATH_INFO: `/` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:07.462045 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 870/1712] 10.222.0.36 () {54 vars in 912 bytes} [Wed Jul 27 10:48:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 10:48:07.472820 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73788725-19c4-4e6d-9102-d371e7627e29 None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:07.473159 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73788725-19c4-4e6d-9102-d371e7627e29 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:07.473376 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73788725-19c4-4e6d-9102-d371e7627e29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:07.518079 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 843/1713] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Jul 27 10:48:07 2022] POST /identity/v3/projects => generated 367 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 10:48:07.524720 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4776587e-c975-479f-8ad4-f92632655171 admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:07.543868 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-73788725-19c4-4e6d-9102-d371e7627e29 None None] MFA Rules not processed for user `c7922d0a1a7c4e808d0b2a80dca0da40`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:48:07.549316 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-73788725-19c4-4e6d-9102-d371e7627e29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:07.779058 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 871/1714] 10.222.0.36 () {58 vars in 1009 bytes} [Wed Jul 27 10:48:07 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 307 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 10:48:07.788324 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73788725-19c4-4e6d-9102-d371e7627e29 None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:07.789890 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-73788725-19c4-4e6d-9102-d371e7627e29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:07.845509 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73788725-19c4-4e6d-9102-d371e7627e29 None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:07.873673 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d40a88-28d0-4158-8281-ed956f179889 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7922d0a1a7c4e808d0b2a80dca0da40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:07.874341 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d40a88-28d0-4158-8281-ed956f179889 service neutron] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:07.874997 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d40a88-28d0-4158-8281-ed956f179889 service neutron] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:07.874997 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d40a88-28d0-4158-8281-ed956f179889 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:07.878006 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4776587e-c975-479f-8ad4-f92632655171 admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:07.915352 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b71384c-d886-4384-94f2-281a38f702a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:07.916538 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b71384c-d886-4384-94f2-281a38f702a7 admin admin] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:07.927367 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d40a88-28d0-4158-8281-ed956f179889 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:07.928909 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d40a88-28d0-4158-8281-ed956f179889 service neutron] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:07.930910 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b71384c-d886-4384-94f2-281a38f702a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:07.938063 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b71384c-d886-4384-94f2-281a38f702a7 admin admin] PATH_INFO: `/v3/users` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:07.939176 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b71384c-d886-4384-94f2-281a38f702a7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:07.949654 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b71384c-d886-4384-94f2-281a38f702a7 admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:07.976609 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 872/1715] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:48:07 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 10:48:08.064454 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 844/1716] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Jul 27 10:48:07 2022] POST /identity/v3/users => generated 290 bytes in 543 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 10:48:08.066009 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d40a88-28d0-4158-8281-ed956f179889 service neutron] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:08.078049 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:48:08.097644 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d40a88-28d0-4158-8281-ed956f179889 service neutron] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:08.113860 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f0cd9d-2d8f-44d8-8e22-bfe7f757a643 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:08.115883 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f0cd9d-2d8f-44d8-8e22-bfe7f757a643 admin admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:08.116951 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f0cd9d-2d8f-44d8-8e22-bfe7f757a643 admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:08.118958 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f0cd9d-2d8f-44d8-8e22-bfe7f757a643 admin admin] PATH_INFO: `/v3/roles` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:08.121398 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f0cd9d-2d8f-44d8-8e22-bfe7f757a643 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:08.127452 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f0cd9d-2d8f-44d8-8e22-bfe7f757a643 admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:08.158630 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 873/1717] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 10:48:08 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 10:48:08.161399 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b71384c-d886-4384-94f2-281a38f702a7 admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:08.196297 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b71384c-d886-4384-94f2-281a38f702a7 admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:08.211586 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69c8d1a-6613-45db-8385-0b0aa6acaa5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:08.212880 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69c8d1a-6613-45db-8385-0b0aa6acaa5a admin admin] REQUEST_METHOD: `PUT` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:08.212880 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69c8d1a-6613-45db-8385-0b0aa6acaa5a admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:08.213256 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69c8d1a-6613-45db-8385-0b0aa6acaa5a admin admin] PATH_INFO: `/v3/projects/e07ad9f0effe451e86ade1cb1a7bf202/users/9501f0f063d642f4b6196a6b3b6738f6/roles/21156560eb6b436bb567a40b4976f3f4` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:08.223440 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac54739-0aac-4ce5-9333-ef5d46eb5b04 None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:08.223757 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac54739-0aac-4ce5-9333-ef5d46eb5b04 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:08.224324 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac54739-0aac-4ce5-9333-ef5d46eb5b04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:08.243505 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69c8d1a-6613-45db-8385-0b0aa6acaa5a admin admin] RBAC: Authorizing `identity:create_grant(project_id=e07ad9f0effe451e86ade1cb1a7bf202, user_id=9501f0f063d642f4b6196a6b3b6738f6, role_id=21156560eb6b436bb567a40b4976f3f4)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:08.246374 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69c8d1a-6613-45db-8385-0b0aa6acaa5a admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:08.257179 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:DELETE=6,keystone:SELECT=89,keystone:INSERT=5 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:48:08.298287 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 845/1718] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 10:48:08 2022] PUT /identity/v3/projects/e07ad9f0effe451e86ade1cb1a7bf202/users/9501f0f063d642f4b6196a6b3b6738f6/roles/21156560eb6b436bb567a40b4976f3f4 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:48:08.302522 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69c8d1a-6613-45db-8385-0b0aa6acaa5a admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:08.453190 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-0ac54739-0aac-4ce5-9333-ef5d46eb5b04 None None] MFA Rules not processed for user `3d9fed24b4ba44b0b5961a603c4362c4`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:48:08.459409 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-0ac54739-0aac-4ce5-9333-ef5d46eb5b04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:08.539839 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69c8d1a-6613-45db-8385-0b0aa6acaa5a admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:08.570307 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd407c21-66da-4ad4-97a2-8a62a9e0070e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:08.571183 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd407c21-66da-4ad4-97a2-8a62a9e0070e admin admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:08.571602 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd407c21-66da-4ad4-97a2-8a62a9e0070e admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:08.572006 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd407c21-66da-4ad4-97a2-8a62a9e0070e admin admin] PATH_INFO: `/v3/roles` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:08.577137 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd407c21-66da-4ad4-97a2-8a62a9e0070e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:08.586789 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd407c21-66da-4ad4-97a2-8a62a9e0070e admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:08.639917 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 874/1719] 10.222.0.36 () {58 vars in 1009 bytes} [Wed Jul 27 10:48:08 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 417 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 10:48:08.645572 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 846/1720] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 10:48:08 2022] GET /identity/v3/roles => generated 1420 bytes in 341 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 10:48:08.646933 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac54739-0aac-4ce5-9333-ef5d46eb5b04 None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:08.702931 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac54739-0aac-4ce5-9333-ef5d46eb5b04 None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:08.725771 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235e7caa-f90e-4e51-91b4-24c7248fe1cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:08.726712 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235e7caa-f90e-4e51-91b4-24c7248fe1cf admin admin] REQUEST_METHOD: `PUT` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:08.729579 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235e7caa-f90e-4e51-91b4-24c7248fe1cf admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:08.730026 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235e7caa-f90e-4e51-91b4-24c7248fe1cf admin admin] PATH_INFO: `/v3/projects/e07ad9f0effe451e86ade1cb1a7bf202/users/9501f0f063d642f4b6196a6b3b6738f6/roles/21156560eb6b436bb567a40b4976f3f4` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:08.740342 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235e7caa-f90e-4e51-91b4-24c7248fe1cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=e07ad9f0effe451e86ade1cb1a7bf202, user_id=9501f0f063d642f4b6196a6b3b6738f6, role_id=21156560eb6b436bb567a40b4976f3f4)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:08.748616 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235e7caa-f90e-4e51-91b4-24c7248fe1cf admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:08.792743 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 875/1721] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 10:48:08 2022] PUT /identity/v3/projects/e07ad9f0effe451e86ade1cb1a7bf202/users/9501f0f063d642f4b6196a6b3b6738f6/roles/21156560eb6b436bb567a40b4976f3f4 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 10:48:08.813631 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d41482-bf6f-4a49-9a1a-de9c14a8308b None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:08.814136 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d41482-bf6f-4a49-9a1a-de9c14a8308b None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:08.814815 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d41482-bf6f-4a49-9a1a-de9c14a8308b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:08.902500 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235e7caa-f90e-4e51-91b4-24c7248fe1cf admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:08.904660 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-235e7caa-f90e-4e51-91b4-24c7248fe1cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:08.940614 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-74d41482-bf6f-4a49-9a1a-de9c14a8308b None None] MFA Rules not processed for user `9501f0f063d642f4b6196a6b3b6738f6`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:48:08.948232 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-74d41482-bf6f-4a49-9a1a-de9c14a8308b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:09.239130 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235e7caa-f90e-4e51-91b4-24c7248fe1cf admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:09.290388 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c27d8e-1f35-4940-a23e-afbd2e8154f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:09.292448 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c27d8e-1f35-4940-a23e-afbd2e8154f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:09.292874 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c27d8e-1f35-4940-a23e-afbd2e8154f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:09.294542 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c27d8e-1f35-4940-a23e-afbd2e8154f4 admin admin] PATH_INFO: `/v3/projects/977cf028e9a1422f96c2088a4db3d8d6` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:09.310207 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c27d8e-1f35-4940-a23e-afbd2e8154f4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=977cf028e9a1422f96c2088a4db3d8d6)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:09.317489 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c27d8e-1f35-4940-a23e-afbd2e8154f4 admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:09.364405 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 847/1722] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:48:08 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 552 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:48:09.369663 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d41482-bf6f-4a49-9a1a-de9c14a8308b None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:09.372752 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-74d41482-bf6f-4a49-9a1a-de9c14a8308b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:09.569006 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-23c27d8e-1f35-4940-a23e-afbd2e8154f4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '977cf028e9a1422f96c2088a4db3d8d6', 'request_id': 'req-23c27d8e-1f35-4940-a23e-afbd2e8154f4'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:09.573084 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=95,keystone:INSERT=1,keystone:DELETE=4 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:48:09.578632 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:48:09.581280 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-23c27d8e-1f35-4940-a23e-afbd2e8154f4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '977cf028e9a1422f96c2088a4db3d8d6', 'request_id': 'req-23c27d8e-1f35-4940-a23e-afbd2e8154f4'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:09.600625 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d41482-bf6f-4a49-9a1a-de9c14a8308b None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:09.604704 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-23c27d8e-1f35-4940-a23e-afbd2e8154f4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '977cf028e9a1422f96c2088a4db3d8d6', 'request_id': 'req-23c27d8e-1f35-4940-a23e-afbd2e8154f4'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:09.629594 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-23c27d8e-1f35-4940-a23e-afbd2e8154f4 admin admin] The token cache is being invalidate because project 977cf028e9a1422f96c2088a4db3d8d6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86827) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 10:48:09.631487 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-23c27d8e-1f35-4940-a23e-afbd2e8154f4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:09.643105 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-23c27d8e-1f35-4940-a23e-afbd2e8154f4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:09.645174 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 876/1723] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Jul 27 10:48:08 2022] DELETE /identity/v3/projects/977cf028e9a1422f96c2088a4db3d8d6 => generated 0 bytes in 743 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:48:09.646165 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c27d8e-1f35-4940-a23e-afbd2e8154f4 admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:09.649812 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-23c27d8e-1f35-4940-a23e-afbd2e8154f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:09.706830 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3238ff49-4bd2-46e9-8e06-e357a4b4f858 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89ac506eefcd4e42a84b44fa15fb3975', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:09.707558 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3238ff49-4bd2-46e9-8e06-e357a4b4f858 service swift] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:09.707914 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3238ff49-4bd2-46e9-8e06-e357a4b4f858 service swift] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:09.709223 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3238ff49-4bd2-46e9-8e06-e357a4b4f858 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:09.711928 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-3238ff49-4bd2-46e9-8e06-e357a4b4f858 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:09.806872 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c27d8e-1f35-4940-a23e-afbd2e8154f4 admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:09.812143 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b0bde4-a88d-4a1c-8d58-983587480e6d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '864d35f268cf45148756109759bb2743', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:09.812790 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b0bde4-a88d-4a1c-8d58-983587480e6d service cinder] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:09.813070 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b0bde4-a88d-4a1c-8d58-983587480e6d service cinder] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:09.813674 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b0bde4-a88d-4a1c-8d58-983587480e6d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:09.820624 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-78b0bde4-a88d-4a1c-8d58-983587480e6d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:09.870071 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3238ff49-4bd2-46e9-8e06-e357a4b4f858 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:09.872610 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3238ff49-4bd2-46e9-8e06-e357a4b4f858 service swift] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:09.899739 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 848/1724] 10.222.0.36 () {60 vars in 1384 bytes} [Wed Jul 27 10:48:09 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 534 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 27 10:48:09.900838 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3238ff49-4bd2-46e9-8e06-e357a4b4f858 service swift] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:09.903922 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-3238ff49-4bd2-46e9-8e06-e357a4b4f858 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:10.162061 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b0bde4-a88d-4a1c-8d58-983587480e6d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:10.171304 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b0bde4-a88d-4a1c-8d58-983587480e6d service cinder] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:10.172717 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3238ff49-4bd2-46e9-8e06-e357a4b4f858 service swift] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:10.198651 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ff5eb3-7561-4eae-a8a4-e00142ff016d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:10.204261 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ff5eb3-7561-4eae-a8a4-e00142ff016d admin admin] REQUEST_METHOD: `DELETE` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:10.205079 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ff5eb3-7561-4eae-a8a4-e00142ff016d admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:10.207676 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ff5eb3-7561-4eae-a8a4-e00142ff016d admin admin] PATH_INFO: `/v3/users/73dd21f6eedd43a492605ad449984d46` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:10.213913 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ff5eb3-7561-4eae-a8a4-e00142ff016d admin admin] RBAC: Authorizing `identity:delete_user(user_id=73dd21f6eedd43a492605ad449984d46)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:10.228732 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ff5eb3-7561-4eae-a8a4-e00142ff016d admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:10.259721 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 877/1725] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:48:09 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 612 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 10:48:10.281242 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b0bde4-a88d-4a1c-8d58-983587480e6d service cinder] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:10.284213 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-78b0bde4-a88d-4a1c-8d58-983587480e6d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:10.357977 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b0bde4-a88d-4a1c-8d58-983587480e6d service cinder] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:10.377122 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285af620-88b9-472d-b680-a5e40e335e6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:10.380098 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285af620-88b9-472d-b680-a5e40e335e6f admin admin] REQUEST_METHOD: `DELETE` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:10.380516 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285af620-88b9-472d-b680-a5e40e335e6f admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:10.381291 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285af620-88b9-472d-b680-a5e40e335e6f admin admin] PATH_INFO: `/v3/projects/e52022826ef84644b89cca2931ac53a1` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:10.385667 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285af620-88b9-472d-b680-a5e40e335e6f admin admin] RBAC: Authorizing `identity:delete_project(project_id=e52022826ef84644b89cca2931ac53a1)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:10.420428 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285af620-88b9-472d-b680-a5e40e335e6f admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:10.485967 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:48:10.521614 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-06ff5eb3-7561-4eae-a8a4-e00142ff016d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '73dd21f6eedd43a492605ad449984d46'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:10.565468 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-06ff5eb3-7561-4eae-a8a4-e00142ff016d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '73dd21f6eedd43a492605ad449984d46'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:10.593698 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-06ff5eb3-7561-4eae-a8a4-e00142ff016d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '73dd21f6eedd43a492605ad449984d46'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:10.594704 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-06ff5eb3-7561-4eae-a8a4-e00142ff016d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '73dd21f6eedd43a492605ad449984d46'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:10.595964 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-06ff5eb3-7561-4eae-a8a4-e00142ff016d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '73dd21f6eedd43a492605ad449984d46'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:10.753122 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 849/1726] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Jul 27 10:48:09 2022] DELETE /identity/v3/users/73dd21f6eedd43a492605ad449984d46 => generated 0 bytes in 853 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:48:10.758056 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-285af620-88b9-472d-b680-a5e40e335e6f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e52022826ef84644b89cca2931ac53a1', 'request_id': 'req-285af620-88b9-472d-b680-a5e40e335e6f'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:10.792536 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-285af620-88b9-472d-b680-a5e40e335e6f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e52022826ef84644b89cca2931ac53a1', 'request_id': 'req-285af620-88b9-472d-b680-a5e40e335e6f'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:10.823695 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-285af620-88b9-472d-b680-a5e40e335e6f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e52022826ef84644b89cca2931ac53a1', 'request_id': 'req-285af620-88b9-472d-b680-a5e40e335e6f'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:10.883974 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-285af620-88b9-472d-b680-a5e40e335e6f admin admin] The token cache is being invalidate because project e52022826ef84644b89cca2931ac53a1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86827) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 10:48:10.884279 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-285af620-88b9-472d-b680-a5e40e335e6f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:10.891067 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-285af620-88b9-472d-b680-a5e40e335e6f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:10.908751 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796bd695-6064-47bc-af3e-780b700f4023 None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:10.908935 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796bd695-6064-47bc-af3e-780b700f4023 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:10.909153 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796bd695-6064-47bc-af3e-780b700f4023 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:10.912838 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 878/1727] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Jul 27 10:48:10 2022] DELETE /identity/v3/projects/e52022826ef84644b89cca2931ac53a1 => generated 0 bytes in 622 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:48:10.982106 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-796bd695-6064-47bc-af3e-780b700f4023 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:48:10.983510 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-796bd695-6064-47bc-af3e-780b700f4023 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:11.321519 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 850/1728] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:48:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 414 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:48:11.325529 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cb778a-4916-41d7-9cc9-df1af9d0ace1 None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:11.325831 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cb778a-4916-41d7-9cc9-df1af9d0ace1 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:11.326147 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cb778a-4916-41d7-9cc9-df1af9d0ace1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:11.407308 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-58cb778a-4916-41d7-9cc9-df1af9d0ace1 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:48:11.412419 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-58cb778a-4916-41d7-9cc9-df1af9d0ace1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:11.446411 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 879/1729] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:48:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:48:11.453062 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796bd695-6064-47bc-af3e-780b700f4023 None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:11.505507 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796bd695-6064-47bc-af3e-780b700f4023 None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:11.542525 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def50a4b-8611-44d6-ab1e-925971041aeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:11.548584 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def50a4b-8611-44d6-ab1e-925971041aeb admin admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:11.551487 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def50a4b-8611-44d6-ab1e-925971041aeb admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:11.552328 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def50a4b-8611-44d6-ab1e-925971041aeb admin admin] PATH_INFO: `/v3/domains` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:11.553998 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def50a4b-8611-44d6-ab1e-925971041aeb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86828) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:48:11.556723 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def50a4b-8611-44d6-ab1e-925971041aeb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:11.565113 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def50a4b-8611-44d6-ab1e-925971041aeb admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:11.601474 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 851/1730] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Jul 27 10:48:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:48:11.602752 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cb778a-4916-41d7-9cc9-df1af9d0ace1 None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:11.619217 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cb778a-4916-41d7-9cc9-df1af9d0ace1 None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:11.651502 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010d5c2f-b04f-4d50-8404-875f83a53b93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:11.671178 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010d5c2f-b04f-4d50-8404-875f83a53b93 admin admin] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:11.671487 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010d5c2f-b04f-4d50-8404-875f83a53b93 admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:11.671776 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010d5c2f-b04f-4d50-8404-875f83a53b93 admin admin] PATH_INFO: `/v3/projects` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:11.672491 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010d5c2f-b04f-4d50-8404-875f83a53b93 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:11.690082 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010d5c2f-b04f-4d50-8404-875f83a53b93 admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:11.716642 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:48:11.819669 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 880/1731] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Jul 27 10:48:11 2022] POST /identity/v3/projects => generated 371 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 10:48:11.821985 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def50a4b-8611-44d6-ab1e-925971041aeb admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:11.838795 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def50a4b-8611-44d6-ab1e-925971041aeb admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:12.099415 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b94fa00-9605-455d-ba35-4559e7ce1528 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:12.103008 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b94fa00-9605-455d-ba35-4559e7ce1528 admin admin] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:12.103302 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b94fa00-9605-455d-ba35-4559e7ce1528 admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:12.103629 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b94fa00-9605-455d-ba35-4559e7ce1528 admin admin] PATH_INFO: `/v3/users` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:12.107914 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b94fa00-9605-455d-ba35-4559e7ce1528 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:12.114881 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b94fa00-9605-455d-ba35-4559e7ce1528 admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:12.183289 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 852/1732] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Jul 27 10:48:11 2022] POST /identity/v3/users => generated 286 bytes in 362 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 10:48:12.187718 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010d5c2f-b04f-4d50-8404-875f83a53b93 admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:12.219782 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010d5c2f-b04f-4d50-8404-875f83a53b93 admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:12.258922 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc898e2-1688-4b9c-85a8-d0a4181d9136 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:12.259974 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc898e2-1688-4b9c-85a8-d0a4181d9136 admin admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:12.260262 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc898e2-1688-4b9c-85a8-d0a4181d9136 admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:12.265175 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc898e2-1688-4b9c-85a8-d0a4181d9136 admin admin] PATH_INFO: `/v3/roles` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:12.268379 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc898e2-1688-4b9c-85a8-d0a4181d9136 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:12.274486 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc898e2-1688-4b9c-85a8-d0a4181d9136 admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:12.325326 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 881/1733] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 10:48:12 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 10:48:12.331411 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b94fa00-9605-455d-ba35-4559e7ce1528 admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:12.362075 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b94fa00-9605-455d-ba35-4559e7ce1528 admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:12.377415 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8957a51-6793-4342-b801-4a28fd2f770f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:12.378328 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8957a51-6793-4342-b801-4a28fd2f770f admin admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:12.379460 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8957a51-6793-4342-b801-4a28fd2f770f admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:12.379754 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8957a51-6793-4342-b801-4a28fd2f770f admin admin] PATH_INFO: `/v3/roles` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:12.388226 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8957a51-6793-4342-b801-4a28fd2f770f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:12.394760 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8957a51-6793-4342-b801-4a28fd2f770f admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:12.449474 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 853/1734] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 10:48:12 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 10:48:12.454315 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc898e2-1688-4b9c-85a8-d0a4181d9136 admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:12.468699 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc898e2-1688-4b9c-85a8-d0a4181d9136 admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:12.486494 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e6c2d5-8a2c-4ca7-8e47-8c92ae19c5db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:12.495645 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e6c2d5-8a2c-4ca7-8e47-8c92ae19c5db admin admin] REQUEST_METHOD: `PUT` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:12.495926 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e6c2d5-8a2c-4ca7-8e47-8c92ae19c5db admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:12.496796 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e6c2d5-8a2c-4ca7-8e47-8c92ae19c5db admin admin] PATH_INFO: `/v3/projects/70a57a7503f74aad9669576dde7a448c/users/aba74e6e2a4e4dcdbe3669ef44dada6e/roles/4c9fc0b3baed441da2126be41cc6b04a` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:12.562699 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e6c2d5-8a2c-4ca7-8e47-8c92ae19c5db admin admin] RBAC: Authorizing `identity:create_grant(project_id=70a57a7503f74aad9669576dde7a448c, user_id=aba74e6e2a4e4dcdbe3669ef44dada6e, role_id=4c9fc0b3baed441da2126be41cc6b04a)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:12.569133 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e6c2d5-8a2c-4ca7-8e47-8c92ae19c5db admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:12.617738 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 882/1735] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 10:48:12 2022] PUT /identity/v3/projects/70a57a7503f74aad9669576dde7a448c/users/aba74e6e2a4e4dcdbe3669ef44dada6e/roles/4c9fc0b3baed441da2126be41cc6b04a => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:48:12.632661 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a1ac77-4e2a-492e-9045-0d09d3733808 None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:12.634416 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a1ac77-4e2a-492e-9045-0d09d3733808 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:12.634870 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a1ac77-4e2a-492e-9045-0d09d3733808 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:12.730874 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-76a1ac77-4e2a-492e-9045-0d09d3733808 None None] MFA Rules not processed for user `aba74e6e2a4e4dcdbe3669ef44dada6e`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:48:12.735238 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-76a1ac77-4e2a-492e-9045-0d09d3733808 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:12.767542 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:48:13.049596 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 854/1736] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:48:12 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 420 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 10:48:13.062351 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e6c2d5-8a2c-4ca7-8e47-8c92ae19c5db admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:13.064590 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-d4e6c2d5-8a2c-4ca7-8e47-8c92ae19c5db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:13.351114 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e6c2d5-8a2c-4ca7-8e47-8c92ae19c5db admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:13.377972 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28512c49-3563-4879-86be-8a8cea7e8791 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e059d8b6c28c4fcba925d05d7c98e4ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:13.378797 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28512c49-3563-4879-86be-8a8cea7e8791 service nova] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:13.379640 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28512c49-3563-4879-86be-8a8cea7e8791 service nova] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:13.380238 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28512c49-3563-4879-86be-8a8cea7e8791 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:13.382701 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a1ac77-4e2a-492e-9045-0d09d3733808 None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:13.385217 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-76a1ac77-4e2a-492e-9045-0d09d3733808 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:13.423387 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28512c49-3563-4879-86be-8a8cea7e8791 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:13.431345 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28512c49-3563-4879-86be-8a8cea7e8791 service nova] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:13.468038 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 883/1737] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:48:13 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 10:48:13.475318 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4ac135-9041-466b-9733-463e63b9c8f2 None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:13.477293 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4ac135-9041-466b-9733-463e63b9c8f2 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:13.480149 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4ac135-9041-466b-9733-463e63b9c8f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:13.558556 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-8b4ac135-9041-466b-9733-463e63b9c8f2 None None] MFA Rules not processed for user `9b094c00959246f5a113f600fb07215d`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:48:13.565928 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-8b4ac135-9041-466b-9733-463e63b9c8f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:13.713518 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a1ac77-4e2a-492e-9045-0d09d3733808 None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:13.743412 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff53143f-960b-4266-926d-e0a820b61fb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:13.745685 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff53143f-960b-4266-926d-e0a820b61fb6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:13.750577 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff53143f-960b-4266-926d-e0a820b61fb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:13.751151 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff53143f-960b-4266-926d-e0a820b61fb6 admin admin] PATH_INFO: `/v3/projects/727403ef1f3d435d9af07604643eebd1` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:13.756847 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff53143f-960b-4266-926d-e0a820b61fb6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=727403ef1f3d435d9af07604643eebd1)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:13.764351 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff53143f-960b-4266-926d-e0a820b61fb6 admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:13.832483 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 884/1738] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:48:13 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 363 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:48:13.838546 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f023512-48b2-4aba-80cb-43a1a9d8a160 None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:13.838986 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f023512-48b2-4aba-80cb-43a1a9d8a160 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:13.840871 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f023512-48b2-4aba-80cb-43a1a9d8a160 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:13.897980 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-7f023512-48b2-4aba-80cb-43a1a9d8a160 None None] MFA Rules not processed for user `c7922d0a1a7c4e808d0b2a80dca0da40`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:48:13.899684 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-7f023512-48b2-4aba-80cb-43a1a9d8a160 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:13.914416 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-ff53143f-960b-4266-926d-e0a820b61fb6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '727403ef1f3d435d9af07604643eebd1', 'request_id': 'req-ff53143f-960b-4266-926d-e0a820b61fb6'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:13.926709 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-ff53143f-960b-4266-926d-e0a820b61fb6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '727403ef1f3d435d9af07604643eebd1', 'request_id': 'req-ff53143f-960b-4266-926d-e0a820b61fb6'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:13.952797 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-ff53143f-960b-4266-926d-e0a820b61fb6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '727403ef1f3d435d9af07604643eebd1', 'request_id': 'req-ff53143f-960b-4266-926d-e0a820b61fb6'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:13.985652 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-ff53143f-960b-4266-926d-e0a820b61fb6 admin admin] The token cache is being invalidate because project 727403ef1f3d435d9af07604643eebd1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86828) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 10:48:13.987941 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-ff53143f-960b-4266-926d-e0a820b61fb6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:13.989491 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-ff53143f-960b-4266-926d-e0a820b61fb6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:13.998529 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 855/1739] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Jul 27 10:48:13 2022] DELETE /identity/v3/projects/727403ef1f3d435d9af07604643eebd1 => generated 0 bytes in 616 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:48:14.004022 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6b682c-9d0e-4fd5-819f-a857b2092faf None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:14.004503 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6b682c-9d0e-4fd5-819f-a857b2092faf None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:14.005945 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6b682c-9d0e-4fd5-819f-a857b2092faf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:14.043018 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:48:14.081461 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-fb6b682c-9d0e-4fd5-819f-a857b2092faf None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:48:14.082541 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-fb6b682c-9d0e-4fd5-819f-a857b2092faf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:14.124773 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 885/1740] 10.222.0.69 () {58 vars in 1015 bytes} [Wed Jul 27 10:48:13 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 291 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:48:14.163253 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f023512-48b2-4aba-80cb-43a1a9d8a160 None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:14.164811 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-7f023512-48b2-4aba-80cb-43a1a9d8a160 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:14.198487 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f023512-48b2-4aba-80cb-43a1a9d8a160 None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:14.219167 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00a80b5-a751-47d9-9cf1-5322e8675cd9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7922d0a1a7c4e808d0b2a80dca0da40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:14.219849 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00a80b5-a751-47d9-9cf1-5322e8675cd9 service neutron] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:14.220142 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00a80b5-a751-47d9-9cf1-5322e8675cd9 service neutron] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:14.220425 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00a80b5-a751-47d9-9cf1-5322e8675cd9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:14.252738 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00a80b5-a751-47d9-9cf1-5322e8675cd9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:14.255258 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:48:14.281956 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00a80b5-a751-47d9-9cf1-5322e8675cd9 service neutron] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:14.316703 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 856/1741] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:48:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 314 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:48:14.337942 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a7809e-5577-4cf4-b7b1-972701730481 None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:14.339759 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a7809e-5577-4cf4-b7b1-972701730481 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:14.340206 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a7809e-5577-4cf4-b7b1-972701730481 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:14.355268 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 886/1742] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:48:14 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 10:48:14.463290 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-83a7809e-5577-4cf4-b7b1-972701730481 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:48:14.488696 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-83a7809e-5577-4cf4-b7b1-972701730481 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:14.525152 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00a80b5-a751-47d9-9cf1-5322e8675cd9 service neutron] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:14.525526 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 857/1743] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:48:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:48:14.548283 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00a80b5-a751-47d9-9cf1-5322e8675cd9 service neutron] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:14.573608 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac02369d-4a39-4edb-af65-9d5c08371e3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:14.574260 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac02369d-4a39-4edb-af65-9d5c08371e3e admin admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:14.574541 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac02369d-4a39-4edb-af65-9d5c08371e3e admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:14.575392 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac02369d-4a39-4edb-af65-9d5c08371e3e admin admin] PATH_INFO: `/v3/domains` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:14.576105 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac02369d-4a39-4edb-af65-9d5c08371e3e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86827) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:48:14.576411 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac02369d-4a39-4edb-af65-9d5c08371e3e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:14.578032 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac02369d-4a39-4edb-af65-9d5c08371e3e admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:14.658815 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 887/1744] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Jul 27 10:48:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:48:14.663498 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a7809e-5577-4cf4-b7b1-972701730481 None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:14.701884 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a7809e-5577-4cf4-b7b1-972701730481 None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:14.716202 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4290cbc-51d6-4887-8f97-2ae0367ee90e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:14.718306 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4290cbc-51d6-4887-8f97-2ae0367ee90e admin admin] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:14.719308 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4290cbc-51d6-4887-8f97-2ae0367ee90e admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:14.720631 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4290cbc-51d6-4887-8f97-2ae0367ee90e admin admin] PATH_INFO: `/v3/projects` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:14.729164 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4290cbc-51d6-4887-8f97-2ae0367ee90e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:14.743143 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4290cbc-51d6-4887-8f97-2ae0367ee90e admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:14.851060 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 858/1745] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Jul 27 10:48:14 2022] POST /identity/v3/projects => generated 391 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 10:48:14.857462 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac02369d-4a39-4edb-af65-9d5c08371e3e admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:14.885844 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac02369d-4a39-4edb-af65-9d5c08371e3e admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:15.041198 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178d380c-c0e2-4cb3-9f85-469e636723b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:15.054925 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178d380c-c0e2-4cb3-9f85-469e636723b5 admin admin] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:15.057779 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178d380c-c0e2-4cb3-9f85-469e636723b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:15.061180 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178d380c-c0e2-4cb3-9f85-469e636723b5 admin admin] PATH_INFO: `/v3/users` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:15.064355 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178d380c-c0e2-4cb3-9f85-469e636723b5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:15.095155 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178d380c-c0e2-4cb3-9f85-469e636723b5 admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:15.218069 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 888/1746] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Jul 27 10:48:14 2022] POST /identity/v3/users => generated 296 bytes in 363 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 10:48:15.222207 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4290cbc-51d6-4887-8f97-2ae0367ee90e admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:15.248040 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4290cbc-51d6-4887-8f97-2ae0367ee90e admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:15.258231 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3d12ea-aba5-48d4-934d-a291e082f73f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:15.259762 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3d12ea-aba5-48d4-934d-a291e082f73f admin admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:15.260531 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3d12ea-aba5-48d4-934d-a291e082f73f admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:15.262705 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3d12ea-aba5-48d4-934d-a291e082f73f admin admin] PATH_INFO: `/v3/roles` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:15.265537 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3d12ea-aba5-48d4-934d-a291e082f73f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:15.269071 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3d12ea-aba5-48d4-934d-a291e082f73f admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:15.304144 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 859/1747] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 10:48:15 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 10:48:15.306146 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178d380c-c0e2-4cb3-9f85-469e636723b5 admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:15.330784 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178d380c-c0e2-4cb3-9f85-469e636723b5 admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:15.339213 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e94f6a-7bf3-49f1-a453-66bcd3280d44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:15.344783 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e94f6a-7bf3-49f1-a453-66bcd3280d44 admin admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:15.345217 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e94f6a-7bf3-49f1-a453-66bcd3280d44 admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:15.345797 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e94f6a-7bf3-49f1-a453-66bcd3280d44 admin admin] PATH_INFO: `/v3/roles` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:15.347382 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e94f6a-7bf3-49f1-a453-66bcd3280d44 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:15.349691 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e94f6a-7bf3-49f1-a453-66bcd3280d44 admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:15.369794 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 889/1748] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 10:48:15 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 10:48:15.374165 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3d12ea-aba5-48d4-934d-a291e082f73f admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:15.401436 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3d12ea-aba5-48d4-934d-a291e082f73f admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:15.436264 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c0461e-8f23-4fcd-b198-23ba30e00f69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:15.437097 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c0461e-8f23-4fcd-b198-23ba30e00f69 admin admin] REQUEST_METHOD: `PUT` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:15.437560 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c0461e-8f23-4fcd-b198-23ba30e00f69 admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:15.437999 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c0461e-8f23-4fcd-b198-23ba30e00f69 admin admin] PATH_INFO: `/v3/projects/6bf07550b3fd4bfcb611a5c9b2eafc33/users/2075ea40d63f4978b41c91d4bbc325dc/roles/4c9fc0b3baed441da2126be41cc6b04a` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:15.482147 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c0461e-8f23-4fcd-b198-23ba30e00f69 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6bf07550b3fd4bfcb611a5c9b2eafc33, user_id=2075ea40d63f4978b41c91d4bbc325dc, role_id=4c9fc0b3baed441da2126be41cc6b04a)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:15.485240 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c0461e-8f23-4fcd-b198-23ba30e00f69 admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:15.509435 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 860/1749] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 10:48:15 2022] PUT /identity/v3/projects/6bf07550b3fd4bfcb611a5c9b2eafc33/users/2075ea40d63f4978b41c91d4bbc325dc/roles/4c9fc0b3baed441da2126be41cc6b04a => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 10:48:15.522786 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26c7607-f54b-43df-ae28-cd6b554df9c2 None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:15.523120 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26c7607-f54b-43df-ae28-cd6b554df9c2 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:15.523375 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26c7607-f54b-43df-ae28-cd6b554df9c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:15.616832 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-f26c7607-f54b-43df-ae28-cd6b554df9c2 None None] MFA Rules not processed for user `2075ea40d63f4978b41c91d4bbc325dc`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:48:15.628604 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-f26c7607-f54b-43df-ae28-cd6b554df9c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:15.763821 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 890/1750] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:48:15 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:48:15.764396 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c0461e-8f23-4fcd-b198-23ba30e00f69 admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:15.865427 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c0461e-8f23-4fcd-b198-23ba30e00f69 admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:15.872602 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c3c2f8-2312-42e2-9e79-83eea59dda61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:15.873472 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c3c2f8-2312-42e2-9e79-83eea59dda61 admin admin] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:15.874025 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c3c2f8-2312-42e2-9e79-83eea59dda61 admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:15.874493 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c3c2f8-2312-42e2-9e79-83eea59dda61 admin admin] PATH_INFO: `/v3/projects` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:15.875581 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c3c2f8-2312-42e2-9e79-83eea59dda61 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:15.879282 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c3c2f8-2312-42e2-9e79-83eea59dda61 admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:15.944503 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26c7607-f54b-43df-ae28-cd6b554df9c2 None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:15.957141 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 861/1751] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Jul 27 10:48:15 2022] POST /identity/v3/projects => generated 391 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 10:48:15.966129 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26c7607-f54b-43df-ae28-cd6b554df9c2 None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:16.012207 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:48:16.073701 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f537bc6-a532-403d-9272-64689dc0d1dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:16.074351 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f537bc6-a532-403d-9272-64689dc0d1dc admin admin] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:16.074783 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f537bc6-a532-403d-9272-64689dc0d1dc admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:16.075114 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f537bc6-a532-403d-9272-64689dc0d1dc admin admin] PATH_INFO: `/v3/users` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:16.075973 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f537bc6-a532-403d-9272-64689dc0d1dc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:16.079413 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f537bc6-a532-403d-9272-64689dc0d1dc admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:16.173477 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 891/1752] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Jul 27 10:48:15 2022] POST /identity/v3/users => generated 302 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 10:48:16.174188 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c3c2f8-2312-42e2-9e79-83eea59dda61 admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:16.194350 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c3c2f8-2312-42e2-9e79-83eea59dda61 admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:16.217182 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c98d48-d003-4013-ad45-0beed76be9e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:16.217846 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c98d48-d003-4013-ad45-0beed76be9e9 admin admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:16.218128 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c98d48-d003-4013-ad45-0beed76be9e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:16.218578 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c98d48-d003-4013-ad45-0beed76be9e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:16.219318 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c98d48-d003-4013-ad45-0beed76be9e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:16.220955 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c98d48-d003-4013-ad45-0beed76be9e9 admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:16.253141 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f537bc6-a532-403d-9272-64689dc0d1dc admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:16.261334 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 862/1753] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 10:48:16 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 10:48:16.297164 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f537bc6-a532-403d-9272-64689dc0d1dc admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:16.324871 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1d44b2-220c-4560-9efa-ee3682a0ea04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:16.326143 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1d44b2-220c-4560-9efa-ee3682a0ea04 admin admin] REQUEST_METHOD: `PUT` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:16.326885 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1d44b2-220c-4560-9efa-ee3682a0ea04 admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:16.327637 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1d44b2-220c-4560-9efa-ee3682a0ea04 admin admin] PATH_INFO: `/v3/projects/1b333cc578d44a2cb514d27097b3c359/users/4efb00e883f2427eaf9e2acf4752b0f5/roles/21156560eb6b436bb567a40b4976f3f4` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:16.372669 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1d44b2-220c-4560-9efa-ee3682a0ea04 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1b333cc578d44a2cb514d27097b3c359, user_id=4efb00e883f2427eaf9e2acf4752b0f5, role_id=21156560eb6b436bb567a40b4976f3f4)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:16.375563 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1d44b2-220c-4560-9efa-ee3682a0ea04 admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:16.407969 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 892/1754] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 10:48:16 2022] PUT /identity/v3/projects/1b333cc578d44a2cb514d27097b3c359/users/4efb00e883f2427eaf9e2acf4752b0f5/roles/21156560eb6b436bb567a40b4976f3f4 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:48:16.412488 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c98d48-d003-4013-ad45-0beed76be9e9 admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:16.426969 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c98d48-d003-4013-ad45-0beed76be9e9 admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:16.553417 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:48:16.573466 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a35317d-6b3c-4d1b-a15c-a34d3871fa98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:16.574114 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a35317d-6b3c-4d1b-a15c-a34d3871fa98 admin admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:16.574394 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a35317d-6b3c-4d1b-a15c-a34d3871fa98 admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:16.574869 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a35317d-6b3c-4d1b-a15c-a34d3871fa98 admin admin] PATH_INFO: `/v3/roles` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:16.575605 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a35317d-6b3c-4d1b-a15c-a34d3871fa98 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:16.577212 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a35317d-6b3c-4d1b-a15c-a34d3871fa98 admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:16.606847 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 863/1755] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 10:48:16 2022] GET /identity/v3/roles => generated 1420 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 10:48:16.611434 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1d44b2-220c-4560-9efa-ee3682a0ea04 admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:16.629300 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1d44b2-220c-4560-9efa-ee3682a0ea04 admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:16.640999 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58402c8f-3ff2-4872-9446-acf1a86cdca2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:16.641599 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58402c8f-3ff2-4872-9446-acf1a86cdca2 admin admin] REQUEST_METHOD: `PUT` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:16.641883 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58402c8f-3ff2-4872-9446-acf1a86cdca2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:16.642197 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58402c8f-3ff2-4872-9446-acf1a86cdca2 admin admin] PATH_INFO: `/v3/projects/1b333cc578d44a2cb514d27097b3c359/users/4efb00e883f2427eaf9e2acf4752b0f5/roles/21156560eb6b436bb567a40b4976f3f4` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:16.645978 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58402c8f-3ff2-4872-9446-acf1a86cdca2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1b333cc578d44a2cb514d27097b3c359, user_id=4efb00e883f2427eaf9e2acf4752b0f5, role_id=21156560eb6b436bb567a40b4976f3f4)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:16.652622 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58402c8f-3ff2-4872-9446-acf1a86cdca2 admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:16.682672 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 893/1756] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 10:48:16 2022] PUT /identity/v3/projects/1b333cc578d44a2cb514d27097b3c359/users/4efb00e883f2427eaf9e2acf4752b0f5/roles/21156560eb6b436bb567a40b4976f3f4 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:48:16.693123 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4410b5-2f73-4141-b46f-66205b0e757b None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:16.693467 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4410b5-2f73-4141-b46f-66205b0e757b None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:16.693752 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4410b5-2f73-4141-b46f-66205b0e757b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:16.799608 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-3a4410b5-2f73-4141-b46f-66205b0e757b None None] MFA Rules not processed for user `4efb00e883f2427eaf9e2acf4752b0f5`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:48:16.805077 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-3a4410b5-2f73-4141-b46f-66205b0e757b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:17.098135 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 864/1757] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:48:16 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 406 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:48:17.345964 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58402c8f-3ff2-4872-9446-acf1a86cdca2 admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:17.348128 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-58402c8f-3ff2-4872-9446-acf1a86cdca2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:17.601495 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58402c8f-3ff2-4872-9446-acf1a86cdca2 admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:17.608868 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4410b5-2f73-4141-b46f-66205b0e757b None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:17.610393 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-3a4410b5-2f73-4141-b46f-66205b0e757b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:17.627192 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7785a12-076f-4212-bcc4-d6d0cb83540d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '864d35f268cf45148756109759bb2743', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:17.628879 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7785a12-076f-4212-bcc4-d6d0cb83540d service cinder] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:17.629886 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7785a12-076f-4212-bcc4-d6d0cb83540d service cinder] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:17.630464 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7785a12-076f-4212-bcc4-d6d0cb83540d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:17.660741 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7785a12-076f-4212-bcc4-d6d0cb83540d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:17.666921 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7785a12-076f-4212-bcc4-d6d0cb83540d service cinder] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:17.724483 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 894/1758] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:48:17 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 10:48:17.813087 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4410b5-2f73-4141-b46f-66205b0e757b None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:17.823171 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906c881f-6420-4a1c-ae4b-d15ddff8f619 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:17.824093 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906c881f-6420-4a1c-ae4b-d15ddff8f619 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:17.824515 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906c881f-6420-4a1c-ae4b-d15ddff8f619 admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:17.825492 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906c881f-6420-4a1c-ae4b-d15ddff8f619 admin admin] PATH_INFO: `/v3/users/0e30b5c2011e40f2aa680f0dee4512a2` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:17.827953 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906c881f-6420-4a1c-ae4b-d15ddff8f619 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0e30b5c2011e40f2aa680f0dee4512a2)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:17.830057 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906c881f-6420-4a1c-ae4b-d15ddff8f619 admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:17.860847 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7785a12-076f-4212-bcc4-d6d0cb83540d service cinder] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:17.872739 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7785a12-076f-4212-bcc4-d6d0cb83540d service cinder] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:17.885624 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b247d6e1-758e-4357-9c7f-3d0d2ae7c7da service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '864d35f268cf45148756109759bb2743', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:17.889592 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b247d6e1-758e-4357-9c7f-3d0d2ae7c7da service cinder] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:17.892806 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b247d6e1-758e-4357-9c7f-3d0d2ae7c7da service cinder] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:17.893619 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b247d6e1-758e-4357-9c7f-3d0d2ae7c7da service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:17.955186 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-906c881f-6420-4a1c-ae4b-d15ddff8f619 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0e30b5c2011e40f2aa680f0dee4512a2'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:17.968845 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-906c881f-6420-4a1c-ae4b-d15ddff8f619 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0e30b5c2011e40f2aa680f0dee4512a2'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:17.981128 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-906c881f-6420-4a1c-ae4b-d15ddff8f619 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0e30b5c2011e40f2aa680f0dee4512a2'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:17.981787 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-906c881f-6420-4a1c-ae4b-d15ddff8f619 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0e30b5c2011e40f2aa680f0dee4512a2'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:17.982437 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-906c881f-6420-4a1c-ae4b-d15ddff8f619 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0e30b5c2011e40f2aa680f0dee4512a2'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:18.014139 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:48:18.034124 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 865/1759] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Jul 27 10:48:17 2022] DELETE /identity/v3/users/0e30b5c2011e40f2aa680f0dee4512a2 => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 10:48:18.036729 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84439b61-1567-490c-9f3c-b41dd6983949 None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:18.037014 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84439b61-1567-490c-9f3c-b41dd6983949 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:18.039668 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84439b61-1567-490c-9f3c-b41dd6983949 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:18.095019 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-84439b61-1567-490c-9f3c-b41dd6983949 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:48:18.097254 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-84439b61-1567-490c-9f3c-b41dd6983949 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:18.167361 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b247d6e1-758e-4357-9c7f-3d0d2ae7c7da service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:18.169305 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b247d6e1-758e-4357-9c7f-3d0d2ae7c7da service cinder] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:18.217508 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 895/1760] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:48:17 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 10:48:18.297950 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 866/1761] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:48:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:48:18.375682 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b247d6e1-758e-4357-9c7f-3d0d2ae7c7da service cinder] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:18.505963 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:48:18.682872 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b247d6e1-758e-4357-9c7f-3d0d2ae7c7da service cinder] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:18.875673 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acdf009-cc43-4da6-862d-89e6af3039f5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7922d0a1a7c4e808d0b2a80dca0da40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:18.876685 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acdf009-cc43-4da6-862d-89e6af3039f5 service neutron] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:18.877611 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acdf009-cc43-4da6-862d-89e6af3039f5 service neutron] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:18.884499 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acdf009-cc43-4da6-862d-89e6af3039f5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:18.912017 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5acdf009-cc43-4da6-862d-89e6af3039f5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:18.920008 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6486c630-d649-4b2d-8c07-456f625cc5e7 None None] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:18.920564 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6486c630-d649-4b2d-8c07-456f625cc5e7 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:18.920564 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6486c630-d649-4b2d-8c07-456f625cc5e7 None None] PATH_INFO: `/` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:18.921509 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5acdf009-cc43-4da6-862d-89e6af3039f5 service neutron] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:18.921927 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 867/1762] 10.222.0.36 () {54 vars in 921 bytes} [Wed Jul 27 10:48:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 10:48:18.956830 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 896/1763] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:48:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 582 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 10:48:19.441868 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a58d63-c7df-47bf-a52c-4aeba4c6ace8 None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:19.442304 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a58d63-c7df-47bf-a52c-4aeba4c6ace8 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:19.442710 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a58d63-c7df-47bf-a52c-4aeba4c6ace8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:19.490726 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-b8a58d63-c7df-47bf-a52c-4aeba4c6ace8 None None] MFA Rules not processed for user `e059d8b6c28c4fcba925d05d7c98e4ac`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:48:19.493606 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-b8a58d63-c7df-47bf-a52c-4aeba4c6ace8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:19.935370 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 868/1764] 10.222.0.36 () {58 vars in 1018 bytes} [Wed Jul 27 10:48:19 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 495 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 10:48:20.233394 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acdf009-cc43-4da6-862d-89e6af3039f5 service neutron] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:20.235245 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-5acdf009-cc43-4da6-862d-89e6af3039f5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:20.279655 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acdf009-cc43-4da6-862d-89e6af3039f5 service neutron] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:20.303259 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b925ea4-2780-4472-89b0-113402abcf4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:20.303929 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b925ea4-2780-4472-89b0-113402abcf4d admin admin] REQUEST_METHOD: `DELETE` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:20.310727 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b925ea4-2780-4472-89b0-113402abcf4d admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:20.311688 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b925ea4-2780-4472-89b0-113402abcf4d admin admin] PATH_INFO: `/v3/users/1c242afc942f4d889e51419710075f91` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:20.318961 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b925ea4-2780-4472-89b0-113402abcf4d admin admin] RBAC: Authorizing `identity:delete_user(user_id=1c242afc942f4d889e51419710075f91)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:20.320336 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: WARNING py.warnings [None req-1b925ea4-2780-4472-89b0-113402abcf4d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 27 10:48:20.320336 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: warnings.warn(msg) Jul 27 10:48:20.320336 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: Jul 27 10:48:20.332567 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b925ea4-2780-4472-89b0-113402abcf4d admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:20.652537 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-1b925ea4-2780-4472-89b0-113402abcf4d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1c242afc942f4d889e51419710075f91'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:20.697147 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-1b925ea4-2780-4472-89b0-113402abcf4d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1c242afc942f4d889e51419710075f91'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:20.716414 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-1b925ea4-2780-4472-89b0-113402abcf4d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1c242afc942f4d889e51419710075f91'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:20.718118 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-1b925ea4-2780-4472-89b0-113402abcf4d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1c242afc942f4d889e51419710075f91'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:20.719311 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-1b925ea4-2780-4472-89b0-113402abcf4d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1c242afc942f4d889e51419710075f91'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:20.813269 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a58d63-c7df-47bf-a52c-4aeba4c6ace8 None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:20.816163 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-b8a58d63-c7df-47bf-a52c-4aeba4c6ace8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:20.856788 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 897/1765] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Jul 27 10:48:20 2022] DELETE /identity/v3/users/1c242afc942f4d889e51419710075f91 => generated 0 bytes in 621 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:48:20.861618 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844d7024-d69d-4ebd-ac15-b1b388daaca1 None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:20.861917 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844d7024-d69d-4ebd-ac15-b1b388daaca1 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:20.864214 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844d7024-d69d-4ebd-ac15-b1b388daaca1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:20.873902 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a58d63-c7df-47bf-a52c-4aeba4c6ace8 None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:20.926441 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-844d7024-d69d-4ebd-ac15-b1b388daaca1 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:48:20.936809 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-844d7024-d69d-4ebd-ac15-b1b388daaca1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:21.103314 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f393b1-1fe9-4dcb-b3f4-b6ab21bacdc1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e059d8b6c28c4fcba925d05d7c98e4ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:21.105618 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f393b1-1fe9-4dcb-b3f4-b6ab21bacdc1 service nova] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:21.107293 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f393b1-1fe9-4dcb-b3f4-b6ab21bacdc1 service nova] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:21.107812 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f393b1-1fe9-4dcb-b3f4-b6ab21bacdc1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:21.115296 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-06f393b1-1fe9-4dcb-b3f4-b6ab21bacdc1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:21.155242 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f393b1-1fe9-4dcb-b3f4-b6ab21bacdc1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:21.161732 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f393b1-1fe9-4dcb-b3f4-b6ab21bacdc1 service nova] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:21.199392 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 869/1766] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:48:20 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 10:48:21.226067 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:48:21.309370 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 898/1767] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:48:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 451 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:48:21.353267 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f393b1-1fe9-4dcb-b3f4-b6ab21bacdc1 service nova] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:21.357290 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-06f393b1-1fe9-4dcb-b3f4-b6ab21bacdc1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:21.423255 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:48:21.645801 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844d7024-d69d-4ebd-ac15-b1b388daaca1 None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:21.651318 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-844d7024-d69d-4ebd-ac15-b1b388daaca1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:21.672155 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f393b1-1fe9-4dcb-b3f4-b6ab21bacdc1 service nova] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:21.679913 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a03077-5307-407b-908e-fe18fc8f0901 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7922d0a1a7c4e808d0b2a80dca0da40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:21.680571 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a03077-5307-407b-908e-fe18fc8f0901 service neutron] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:21.680858 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a03077-5307-407b-908e-fe18fc8f0901 service neutron] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:21.681136 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a03077-5307-407b-908e-fe18fc8f0901 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:21.699072 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844d7024-d69d-4ebd-ac15-b1b388daaca1 None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:21.725336 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630c08af-ff99-4ce5-9e5b-34f2f605d917 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:21.730582 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630c08af-ff99-4ce5-9e5b-34f2f605d917 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:21.733323 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a03077-5307-407b-908e-fe18fc8f0901 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:21.735656 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630c08af-ff99-4ce5-9e5b-34f2f605d917 admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:21.735895 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a03077-5307-407b-908e-fe18fc8f0901 service neutron] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:21.736061 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630c08af-ff99-4ce5-9e5b-34f2f605d917 admin admin] PATH_INFO: `/v3/users/aba74e6e2a4e4dcdbe3669ef44dada6e` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:21.742840 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630c08af-ff99-4ce5-9e5b-34f2f605d917 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aba74e6e2a4e4dcdbe3669ef44dada6e)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:21.755039 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630c08af-ff99-4ce5-9e5b-34f2f605d917 admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:21.762992 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 870/1768] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:48:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 10:48:21.888326 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-630c08af-ff99-4ce5-9e5b-34f2f605d917 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aba74e6e2a4e4dcdbe3669ef44dada6e'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:21.905694 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-630c08af-ff99-4ce5-9e5b-34f2f605d917 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aba74e6e2a4e4dcdbe3669ef44dada6e'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:21.943065 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-630c08af-ff99-4ce5-9e5b-34f2f605d917 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aba74e6e2a4e4dcdbe3669ef44dada6e'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:21.944997 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-630c08af-ff99-4ce5-9e5b-34f2f605d917 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aba74e6e2a4e4dcdbe3669ef44dada6e'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:21.946958 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-630c08af-ff99-4ce5-9e5b-34f2f605d917 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aba74e6e2a4e4dcdbe3669ef44dada6e'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:22.106922 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 899/1769] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Jul 27 10:48:21 2022] DELETE /identity/v3/users/aba74e6e2a4e4dcdbe3669ef44dada6e => generated 0 bytes in 462 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:48:22.113422 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1561868f-7ceb-49ad-a919-b9e7ac104c24 None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:22.114104 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1561868f-7ceb-49ad-a919-b9e7ac104c24 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:22.114104 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1561868f-7ceb-49ad-a919-b9e7ac104c24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:22.173611 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-1561868f-7ceb-49ad-a919-b9e7ac104c24 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:48:22.177788 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-1561868f-7ceb-49ad-a919-b9e7ac104c24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:22.353415 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 871/1770] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:48:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:48:22.574020 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630c08af-ff99-4ce5-9e5b-34f2f605d917 admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:22.577819 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-630c08af-ff99-4ce5-9e5b-34f2f605d917 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:22.884600 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630c08af-ff99-4ce5-9e5b-34f2f605d917 admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:22.900096 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd2f1bc-dbdf-41a6-adca-e60d6645706f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7922d0a1a7c4e808d0b2a80dca0da40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:22.901228 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd2f1bc-dbdf-41a6-adca-e60d6645706f service neutron] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:22.901228 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd2f1bc-dbdf-41a6-adca-e60d6645706f service neutron] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:22.903736 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd2f1bc-dbdf-41a6-adca-e60d6645706f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:22.995618 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd2f1bc-dbdf-41a6-adca-e60d6645706f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:23.007928 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd2f1bc-dbdf-41a6-adca-e60d6645706f service neutron] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:23.034337 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 900/1771] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:48:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 461 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 10:48:23.392752 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1561868f-7ceb-49ad-a919-b9e7ac104c24 None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:23.395008 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-1561868f-7ceb-49ad-a919-b9e7ac104c24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:23.428325 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1561868f-7ceb-49ad-a919-b9e7ac104c24 None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:23.451862 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46679349-50c5-4338-a3dd-2503069ad876 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:23.462560 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46679349-50c5-4338-a3dd-2503069ad876 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:23.462880 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46679349-50c5-4338-a3dd-2503069ad876 admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:23.463166 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46679349-50c5-4338-a3dd-2503069ad876 admin admin] PATH_INFO: `/v3/projects/41bc095eb14146d9b1bcbc6b6e12a28e` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:23.464996 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46679349-50c5-4338-a3dd-2503069ad876 admin admin] RBAC: Authorizing `identity:delete_project(project_id=41bc095eb14146d9b1bcbc6b6e12a28e)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:23.486952 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46679349-50c5-4338-a3dd-2503069ad876 admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:23.596088 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-46679349-50c5-4338-a3dd-2503069ad876 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '41bc095eb14146d9b1bcbc6b6e12a28e', 'request_id': 'req-46679349-50c5-4338-a3dd-2503069ad876'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:23.614088 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-46679349-50c5-4338-a3dd-2503069ad876 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '41bc095eb14146d9b1bcbc6b6e12a28e', 'request_id': 'req-46679349-50c5-4338-a3dd-2503069ad876'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:23.633252 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-46679349-50c5-4338-a3dd-2503069ad876 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '41bc095eb14146d9b1bcbc6b6e12a28e', 'request_id': 'req-46679349-50c5-4338-a3dd-2503069ad876'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:23.662648 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-46679349-50c5-4338-a3dd-2503069ad876 admin admin] The token cache is being invalidate because project 41bc095eb14146d9b1bcbc6b6e12a28e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86828) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 10:48:23.662966 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-46679349-50c5-4338-a3dd-2503069ad876 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:23.669203 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-46679349-50c5-4338-a3dd-2503069ad876 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:23.671066 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 872/1772] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Jul 27 10:48:23 2022] DELETE /identity/v3/projects/41bc095eb14146d9b1bcbc6b6e12a28e => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:48:23.675199 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd2f1bc-dbdf-41a6-adca-e60d6645706f service neutron] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:23.698942 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd2f1bc-dbdf-41a6-adca-e60d6645706f service neutron] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:23.727833 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46679349-50c5-4338-a3dd-2503069ad876 admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:23.732325 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-46679349-50c5-4338-a3dd-2503069ad876 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:23.875334 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:48:23.895072 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a7bfda-66ca-460f-a6fd-5e6151a6bce8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:23.895797 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a7bfda-66ca-460f-a6fd-5e6151a6bce8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:23.896090 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a7bfda-66ca-460f-a6fd-5e6151a6bce8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:23.896376 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a7bfda-66ca-460f-a6fd-5e6151a6bce8 admin admin] PATH_INFO: `/v3/users/9501f0f063d642f4b6196a6b3b6738f6` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:23.904271 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a7bfda-66ca-460f-a6fd-5e6151a6bce8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9501f0f063d642f4b6196a6b3b6738f6)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:23.907793 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a7bfda-66ca-460f-a6fd-5e6151a6bce8 admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:23.931923 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46679349-50c5-4338-a3dd-2503069ad876 admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:23.945292 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e03822-f1e6-4fd7-8851-a5f40142819a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:23.946050 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e03822-f1e6-4fd7-8851-a5f40142819a admin admin] REQUEST_METHOD: `DELETE` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:23.946337 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e03822-f1e6-4fd7-8851-a5f40142819a admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:23.946637 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e03822-f1e6-4fd7-8851-a5f40142819a admin admin] PATH_INFO: `/v3/projects/3ded08acf5604d729d22c167e59953cb` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:23.947642 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:48:23.956869 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e03822-f1e6-4fd7-8851-a5f40142819a admin admin] RBAC: Authorizing `identity:delete_project(project_id=3ded08acf5604d729d22c167e59953cb)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:23.967911 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e03822-f1e6-4fd7-8851-a5f40142819a admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:24.084622 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-e5a7bfda-66ca-460f-a6fd-5e6151a6bce8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9501f0f063d642f4b6196a6b3b6738f6'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:24.107967 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-e5a7bfda-66ca-460f-a6fd-5e6151a6bce8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9501f0f063d642f4b6196a6b3b6738f6'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:24.124954 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-e5a7bfda-66ca-460f-a6fd-5e6151a6bce8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9501f0f063d642f4b6196a6b3b6738f6'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:24.126238 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-e5a7bfda-66ca-460f-a6fd-5e6151a6bce8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9501f0f063d642f4b6196a6b3b6738f6'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:24.127720 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-e5a7bfda-66ca-460f-a6fd-5e6151a6bce8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9501f0f063d642f4b6196a6b3b6738f6'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:24.142088 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-73e03822-f1e6-4fd7-8851-a5f40142819a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3ded08acf5604d729d22c167e59953cb', 'request_id': 'req-73e03822-f1e6-4fd7-8851-a5f40142819a'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:24.159940 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-73e03822-f1e6-4fd7-8851-a5f40142819a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3ded08acf5604d729d22c167e59953cb', 'request_id': 'req-73e03822-f1e6-4fd7-8851-a5f40142819a'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:24.169042 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 901/1773] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Jul 27 10:48:23 2022] DELETE /identity/v3/users/9501f0f063d642f4b6196a6b3b6738f6 => generated 0 bytes in 494 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:48:24.171070 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f05af13-0cff-4ed2-aaff-b2e29f6bb58f None None] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:24.172075 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f05af13-0cff-4ed2-aaff-b2e29f6bb58f None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:24.172752 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f05af13-0cff-4ed2-aaff-b2e29f6bb58f None None] PATH_INFO: `/` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:24.174528 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-73e03822-f1e6-4fd7-8851-a5f40142819a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3ded08acf5604d729d22c167e59953cb', 'request_id': 'req-73e03822-f1e6-4fd7-8851-a5f40142819a'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:24.180445 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 902/1774] 10.222.0.36 () {54 vars in 921 bytes} [Wed Jul 27 10:48:24 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 10:48:24.188610 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-73e03822-f1e6-4fd7-8851-a5f40142819a admin admin] The token cache is being invalidate because project 3ded08acf5604d729d22c167e59953cb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86828) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 10:48:24.189212 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-73e03822-f1e6-4fd7-8851-a5f40142819a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:24.190112 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-73e03822-f1e6-4fd7-8851-a5f40142819a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:24.192039 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 873/1775] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Jul 27 10:48:23 2022] DELETE /identity/v3/projects/3ded08acf5604d729d22c167e59953cb => generated 0 bytes in 465 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 10:48:24.196515 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f05af13-0cff-4ed2-aaff-b2e29f6bb58f None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:24.198344 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-7f05af13-0cff-4ed2-aaff-b2e29f6bb58f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:24.230808 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186e19c4-067c-4f37-9a9a-06aff0b4229f None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:24.231184 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186e19c4-067c-4f37-9a9a-06aff0b4229f None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:24.231380 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186e19c4-067c-4f37-9a9a-06aff0b4229f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:24.258287 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-186e19c4-067c-4f37-9a9a-06aff0b4229f None None] MFA Rules not processed for user `e059d8b6c28c4fcba925d05d7c98e4ac`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:48:24.260390 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-186e19c4-067c-4f37-9a9a-06aff0b4229f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:24.353369 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f05af13-0cff-4ed2-aaff-b2e29f6bb58f None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:24.362753 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ababa591-a849-4405-b361-d886e351ddf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:24.363942 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ababa591-a849-4405-b361-d886e351ddf0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:24.364564 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ababa591-a849-4405-b361-d886e351ddf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:24.365144 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ababa591-a849-4405-b361-d886e351ddf0 admin admin] PATH_INFO: `/v3/users/7c9b20b42773416fbfd9a19306488933` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:24.368083 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ababa591-a849-4405-b361-d886e351ddf0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7c9b20b42773416fbfd9a19306488933)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:24.371954 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ababa591-a849-4405-b361-d886e351ddf0 admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:24.384700 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 874/1776] 10.222.0.36 () {58 vars in 1018 bytes} [Wed Jul 27 10:48:24 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:48:24.439173 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186e19c4-067c-4f37-9a9a-06aff0b4229f None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:24.440939 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-186e19c4-067c-4f37-9a9a-06aff0b4229f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:24.467890 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-ababa591-a849-4405-b361-d886e351ddf0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7c9b20b42773416fbfd9a19306488933'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:24.477442 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-ababa591-a849-4405-b361-d886e351ddf0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7c9b20b42773416fbfd9a19306488933'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:24.479233 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186e19c4-067c-4f37-9a9a-06aff0b4229f None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:24.492377 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-ababa591-a849-4405-b361-d886e351ddf0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7c9b20b42773416fbfd9a19306488933'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:24.493367 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-ababa591-a849-4405-b361-d886e351ddf0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7c9b20b42773416fbfd9a19306488933'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:24.494504 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-ababa591-a849-4405-b361-d886e351ddf0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7c9b20b42773416fbfd9a19306488933'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:24.509792 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a2449e-3358-4821-a1ac-7c33343fe992 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e059d8b6c28c4fcba925d05d7c98e4ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:24.514252 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a2449e-3358-4821-a1ac-7c33343fe992 service nova] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:24.515114 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a2449e-3358-4821-a1ac-7c33343fe992 service nova] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:24.515776 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a2449e-3358-4821-a1ac-7c33343fe992 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:24.518889 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-d7a2449e-3358-4821-a1ac-7c33343fe992 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:24.526234 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 903/1777] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Jul 27 10:48:24 2022] DELETE /identity/v3/users/7c9b20b42773416fbfd9a19306488933 => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:48:24.545929 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a2449e-3358-4821-a1ac-7c33343fe992 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:24.548794 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a2449e-3358-4821-a1ac-7c33343fe992 service nova] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:24.574246 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 875/1778] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:48:24 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 10:48:24.592944 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ababa591-a849-4405-b361-d886e351ddf0 admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:24.594665 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-ababa591-a849-4405-b361-d886e351ddf0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:24.607977 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:48:24.694003 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ababa591-a849-4405-b361-d886e351ddf0 admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:24.699066 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0500a1d8-a408-4b52-8497-3df05abbce9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:24.700387 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0500a1d8-a408-4b52-8497-3df05abbce9e admin admin] REQUEST_METHOD: `DELETE` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:24.700894 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0500a1d8-a408-4b52-8497-3df05abbce9e admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:24.701419 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0500a1d8-a408-4b52-8497-3df05abbce9e admin admin] PATH_INFO: `/v3/users/2075ea40d63f4978b41c91d4bbc325dc` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:24.702711 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0500a1d8-a408-4b52-8497-3df05abbce9e admin admin] RBAC: Authorizing `identity:delete_user(user_id=2075ea40d63f4978b41c91d4bbc325dc)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:24.704668 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0500a1d8-a408-4b52-8497-3df05abbce9e admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:24.769789 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-0500a1d8-a408-4b52-8497-3df05abbce9e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2075ea40d63f4978b41c91d4bbc325dc'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:24.780050 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-0500a1d8-a408-4b52-8497-3df05abbce9e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2075ea40d63f4978b41c91d4bbc325dc'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:24.790515 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-0500a1d8-a408-4b52-8497-3df05abbce9e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2075ea40d63f4978b41c91d4bbc325dc'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:24.791107 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-0500a1d8-a408-4b52-8497-3df05abbce9e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2075ea40d63f4978b41c91d4bbc325dc'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:24.792130 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-0500a1d8-a408-4b52-8497-3df05abbce9e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2075ea40d63f4978b41c91d4bbc325dc'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:24.818065 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 904/1779] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Jul 27 10:48:24 2022] DELETE /identity/v3/users/2075ea40d63f4978b41c91d4bbc325dc => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:48:24.820110 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6728903-f5be-4e57-86bf-45bcd0308126 None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:24.820458 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6728903-f5be-4e57-86bf-45bcd0308126 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:24.820741 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6728903-f5be-4e57-86bf-45bcd0308126 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:24.852254 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-c6728903-f5be-4e57-86bf-45bcd0308126 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:48:24.854829 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-c6728903-f5be-4e57-86bf-45bcd0308126 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:24.917597 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:48:24.943272 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0500a1d8-a408-4b52-8497-3df05abbce9e admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:24.944813 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-0500a1d8-a408-4b52-8497-3df05abbce9e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:24.954510 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 876/1780] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:48:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:48:24.964848 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6728903-f5be-4e57-86bf-45bcd0308126 None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:24.966919 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-c6728903-f5be-4e57-86bf-45bcd0308126 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:24.975714 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0500a1d8-a408-4b52-8497-3df05abbce9e admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:24.981954 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96886fe3-e695-430b-bffb-ef8ac08fd387 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:24.982721 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96886fe3-e695-430b-bffb-ef8ac08fd387 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:24.982937 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96886fe3-e695-430b-bffb-ef8ac08fd387 admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:24.983216 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96886fe3-e695-430b-bffb-ef8ac08fd387 admin admin] PATH_INFO: `/v3/projects/46269f939e7e47158ea3d50d0e9eee4a` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:24.984418 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96886fe3-e695-430b-bffb-ef8ac08fd387 admin admin] RBAC: Authorizing `identity:delete_project(project_id=46269f939e7e47158ea3d50d0e9eee4a)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:24.986340 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96886fe3-e695-430b-bffb-ef8ac08fd387 admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:25.061078 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6728903-f5be-4e57-86bf-45bcd0308126 None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:25.064312 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-96886fe3-e695-430b-bffb-ef8ac08fd387 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '46269f939e7e47158ea3d50d0e9eee4a', 'request_id': 'req-96886fe3-e695-430b-bffb-ef8ac08fd387'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:25.081091 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-96886fe3-e695-430b-bffb-ef8ac08fd387 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '46269f939e7e47158ea3d50d0e9eee4a', 'request_id': 'req-96886fe3-e695-430b-bffb-ef8ac08fd387'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:25.103610 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-96886fe3-e695-430b-bffb-ef8ac08fd387 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '46269f939e7e47158ea3d50d0e9eee4a', 'request_id': 'req-96886fe3-e695-430b-bffb-ef8ac08fd387'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:25.122212 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-96886fe3-e695-430b-bffb-ef8ac08fd387 admin admin] The token cache is being invalidate because project 46269f939e7e47158ea3d50d0e9eee4a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86827) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 10:48:25.123493 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-96886fe3-e695-430b-bffb-ef8ac08fd387 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:25.124263 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-96886fe3-e695-430b-bffb-ef8ac08fd387 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:25.125941 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 905/1781] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Jul 27 10:48:24 2022] DELETE /identity/v3/projects/46269f939e7e47158ea3d50d0e9eee4a => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 10:48:25.130728 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96886fe3-e695-430b-bffb-ef8ac08fd387 admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:25.130728 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-96886fe3-e695-430b-bffb-ef8ac08fd387 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:25.135470 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93440f53-f23a-4524-bde4-49fefd377d32 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7922d0a1a7c4e808d0b2a80dca0da40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:25.136375 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93440f53-f23a-4524-bde4-49fefd377d32 service neutron] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:25.137212 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93440f53-f23a-4524-bde4-49fefd377d32 service neutron] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:25.137875 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93440f53-f23a-4524-bde4-49fefd377d32 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:25.140511 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-93440f53-f23a-4524-bde4-49fefd377d32 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:25.163279 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:48:25.234648 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93440f53-f23a-4524-bde4-49fefd377d32 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:25.236308 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93440f53-f23a-4524-bde4-49fefd377d32 service neutron] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:25.237307 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96886fe3-e695-430b-bffb-ef8ac08fd387 admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:25.243703 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806a773e-9f1d-4887-953c-8d0a318772db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:25.244370 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806a773e-9f1d-4887-953c-8d0a318772db admin admin] REQUEST_METHOD: `DELETE` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:25.244803 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806a773e-9f1d-4887-953c-8d0a318772db admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:25.245301 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806a773e-9f1d-4887-953c-8d0a318772db admin admin] PATH_INFO: `/v3/projects/70a57a7503f74aad9669576dde7a448c` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:25.248647 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806a773e-9f1d-4887-953c-8d0a318772db admin admin] RBAC: Authorizing `identity:delete_project(project_id=70a57a7503f74aad9669576dde7a448c)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:25.249798 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806a773e-9f1d-4887-953c-8d0a318772db admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:25.251236 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 877/1782] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:48:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 287 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 10:48:25.252587 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93440f53-f23a-4524-bde4-49fefd377d32 service neutron] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:25.254591 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-93440f53-f23a-4524-bde4-49fefd377d32 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:25.269428 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93440f53-f23a-4524-bde4-49fefd377d32 service neutron] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:25.275758 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2f8d3a-c31f-4b49-9eea-9692cc88a546 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:25.276744 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2f8d3a-c31f-4b49-9eea-9692cc88a546 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:25.276940 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2f8d3a-c31f-4b49-9eea-9692cc88a546 admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:25.277321 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2f8d3a-c31f-4b49-9eea-9692cc88a546 admin admin] PATH_INFO: `/v3/projects/e07ad9f0effe451e86ade1cb1a7bf202` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:25.278706 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2f8d3a-c31f-4b49-9eea-9692cc88a546 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e07ad9f0effe451e86ade1cb1a7bf202)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:25.280741 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2f8d3a-c31f-4b49-9eea-9692cc88a546 admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:25.339811 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-806a773e-9f1d-4887-953c-8d0a318772db admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '70a57a7503f74aad9669576dde7a448c', 'request_id': 'req-806a773e-9f1d-4887-953c-8d0a318772db'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:25.343615 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-2a2f8d3a-c31f-4b49-9eea-9692cc88a546 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e07ad9f0effe451e86ade1cb1a7bf202', 'request_id': 'req-2a2f8d3a-c31f-4b49-9eea-9692cc88a546'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:25.346022 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:48:25.351088 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-806a773e-9f1d-4887-953c-8d0a318772db admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '70a57a7503f74aad9669576dde7a448c', 'request_id': 'req-806a773e-9f1d-4887-953c-8d0a318772db'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:25.363646 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-806a773e-9f1d-4887-953c-8d0a318772db admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '70a57a7503f74aad9669576dde7a448c', 'request_id': 'req-806a773e-9f1d-4887-953c-8d0a318772db'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:25.370482 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-2a2f8d3a-c31f-4b49-9eea-9692cc88a546 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e07ad9f0effe451e86ade1cb1a7bf202', 'request_id': 'req-2a2f8d3a-c31f-4b49-9eea-9692cc88a546'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:25.391463 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-2a2f8d3a-c31f-4b49-9eea-9692cc88a546 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e07ad9f0effe451e86ade1cb1a7bf202', 'request_id': 'req-2a2f8d3a-c31f-4b49-9eea-9692cc88a546'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:25.393582 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-806a773e-9f1d-4887-953c-8d0a318772db admin admin] The token cache is being invalidate because project 70a57a7503f74aad9669576dde7a448c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86827) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 10:48:25.394359 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-806a773e-9f1d-4887-953c-8d0a318772db admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:25.395543 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-806a773e-9f1d-4887-953c-8d0a318772db admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:25.398345 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 906/1783] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Jul 27 10:48:25 2022] DELETE /identity/v3/projects/70a57a7503f74aad9669576dde7a448c => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:48:25.400181 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7022572-176b-45b1-8c45-dbd6fe09318a None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:25.400655 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7022572-176b-45b1-8c45-dbd6fe09318a None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:25.401220 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7022572-176b-45b1-8c45-dbd6fe09318a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:25.408756 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-2a2f8d3a-c31f-4b49-9eea-9692cc88a546 admin admin] The token cache is being invalidate because project e07ad9f0effe451e86ade1cb1a7bf202 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86828) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 10:48:25.409276 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-2a2f8d3a-c31f-4b49-9eea-9692cc88a546 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:25.410194 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-2a2f8d3a-c31f-4b49-9eea-9692cc88a546 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:25.413017 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 878/1784] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Jul 27 10:48:25 2022] DELETE /identity/v3/projects/e07ad9f0effe451e86ade1cb1a7bf202 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:48:25.416030 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162aa11b-f060-47d4-b38d-2c8ff1a5a2db None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:25.416538 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162aa11b-f060-47d4-b38d-2c8ff1a5a2db None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:25.416897 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162aa11b-f060-47d4-b38d-2c8ff1a5a2db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:25.444667 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-a7022572-176b-45b1-8c45-dbd6fe09318a None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:48:25.446846 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-a7022572-176b-45b1-8c45-dbd6fe09318a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:25.451606 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-162aa11b-f060-47d4-b38d-2c8ff1a5a2db None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:48:25.453940 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-162aa11b-f060-47d4-b38d-2c8ff1a5a2db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:25.553079 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 907/1785] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:48:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:48:25.561612 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ec72b1-5bc9-4d15-83ef-14e30ba55c81 None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:25.561815 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ec72b1-5bc9-4d15-83ef-14e30ba55c81 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:25.562094 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ec72b1-5bc9-4d15-83ef-14e30ba55c81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:25.601762 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 879/1786] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:48:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:48:25.601762 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5747fe-cd13-48bb-99a6-ff1f9d716729 None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:25.601762 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5747fe-cd13-48bb-99a6-ff1f9d716729 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:25.601762 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5747fe-cd13-48bb-99a6-ff1f9d716729 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:25.603248 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-02ec72b1-5bc9-4d15-83ef-14e30ba55c81 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:48:25.604526 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-02ec72b1-5bc9-4d15-83ef-14e30ba55c81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:25.616300 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 908/1787] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:48:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:48:25.617674 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4c17da-60f8-444f-adb6-b4c517c922a5 None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:25.617963 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4c17da-60f8-444f-adb6-b4c517c922a5 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:25.618523 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4c17da-60f8-444f-adb6-b4c517c922a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:25.629504 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-0a5747fe-cd13-48bb-99a6-ff1f9d716729 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:48:25.631445 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-0a5747fe-cd13-48bb-99a6-ff1f9d716729 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:25.641737 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 880/1788] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:48:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:48:25.643904 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-6f4c17da-60f8-444f-adb6-b4c517c922a5 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:48:25.644194 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10dfe79-baf2-4bcd-844a-4d8aa0384027 None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:25.644661 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10dfe79-baf2-4bcd-844a-4d8aa0384027 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:25.645129 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10dfe79-baf2-4bcd-844a-4d8aa0384027 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:25.646920 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-6f4c17da-60f8-444f-adb6-b4c517c922a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:25.657704 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 909/1789] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:48:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:48:25.661593 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4c17da-60f8-444f-adb6-b4c517c922a5 None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:25.680259 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4c17da-60f8-444f-adb6-b4c517c922a5 None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:25.685803 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5116807-4687-4c2b-b0c8-c1561a9a45f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:25.686466 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5116807-4687-4c2b-b0c8-c1561a9a45f9 admin admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:25.686799 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5116807-4687-4c2b-b0c8-c1561a9a45f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:25.687076 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5116807-4687-4c2b-b0c8-c1561a9a45f9 admin admin] PATH_INFO: `/v3/domains` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:25.687809 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5116807-4687-4c2b-b0c8-c1561a9a45f9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86827) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:48:25.688118 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5116807-4687-4c2b-b0c8-c1561a9a45f9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:25.689840 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5116807-4687-4c2b-b0c8-c1561a9a45f9 admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:25.690191 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-a10dfe79-baf2-4bcd-844a-4d8aa0384027 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:48:25.691790 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-a10dfe79-baf2-4bcd-844a-4d8aa0384027 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:25.702151 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 881/1790] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:48:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:48:25.703123 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10dfe79-baf2-4bcd-844a-4d8aa0384027 None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:25.710088 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 910/1791] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Jul 27 10:48:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:48:25.711339 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5116807-4687-4c2b-b0c8-c1561a9a45f9 admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:25.724802 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5116807-4687-4c2b-b0c8-c1561a9a45f9 admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:25.730252 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c460044-2ade-4d3f-ada9-5540af77f3c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:25.731002 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c460044-2ade-4d3f-ada9-5540af77f3c1 admin admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:25.731345 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c460044-2ade-4d3f-ada9-5540af77f3c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:25.731666 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10dfe79-baf2-4bcd-844a-4d8aa0384027 None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:25.731787 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c460044-2ade-4d3f-ada9-5540af77f3c1 admin admin] PATH_INFO: `/v3/domains` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:25.732607 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c460044-2ade-4d3f-ada9-5540af77f3c1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86827) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:48:25.732977 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c460044-2ade-4d3f-ada9-5540af77f3c1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:25.734587 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c460044-2ade-4d3f-ada9-5540af77f3c1 admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:25.744492 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e15599-2313-4749-b956-46e46e7f0f03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:25.745148 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e15599-2313-4749-b956-46e46e7f0f03 admin admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:25.745446 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e15599-2313-4749-b956-46e46e7f0f03 admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:25.745715 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e15599-2313-4749-b956-46e46e7f0f03 admin admin] PATH_INFO: `/v3/domains` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:25.746621 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e15599-2313-4749-b956-46e46e7f0f03 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86828) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:48:25.747058 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e15599-2313-4749-b956-46e46e7f0f03 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:25.748694 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e15599-2313-4749-b956-46e46e7f0f03 admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:25.752748 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 911/1792] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Jul 27 10:48:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:48:25.752748 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c460044-2ade-4d3f-ada9-5540af77f3c1 admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:25.763451 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 882/1793] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Jul 27 10:48:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:48:25.764314 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e15599-2313-4749-b956-46e46e7f0f03 admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:25.764733 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:48:25.766085 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-f1e15599-2313-4749-b956-46e46e7f0f03 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:25.770776 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c460044-2ade-4d3f-ada9-5540af77f3c1 admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:25.779129 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ea6f08-6ae8-448c-b182-14db7a3facd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:25.779850 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ea6f08-6ae8-448c-b182-14db7a3facd0 admin admin] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:25.780329 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ea6f08-6ae8-448c-b182-14db7a3facd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:25.780551 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e15599-2313-4749-b956-46e46e7f0f03 admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:25.780670 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ea6f08-6ae8-448c-b182-14db7a3facd0 admin admin] PATH_INFO: `/v3/projects` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:25.781864 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ea6f08-6ae8-448c-b182-14db7a3facd0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:25.784202 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ea6f08-6ae8-448c-b182-14db7a3facd0 admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:25.785505 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68347ec-126b-46bd-be76-b4e2b9fbcb33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:25.786246 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68347ec-126b-46bd-be76-b4e2b9fbcb33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:25.786645 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68347ec-126b-46bd-be76-b4e2b9fbcb33 admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:25.786988 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68347ec-126b-46bd-be76-b4e2b9fbcb33 admin admin] PATH_INFO: `/v3/projects/6bf07550b3fd4bfcb611a5c9b2eafc33` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:25.788240 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68347ec-126b-46bd-be76-b4e2b9fbcb33 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6bf07550b3fd4bfcb611a5c9b2eafc33)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:25.790407 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68347ec-126b-46bd-be76-b4e2b9fbcb33 admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:25.815482 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 912/1794] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Jul 27 10:48:25 2022] POST /identity/v3/projects => generated 379 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 10:48:25.816199 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ea6f08-6ae8-448c-b182-14db7a3facd0 admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:25.843665 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-a68347ec-126b-46bd-be76-b4e2b9fbcb33 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6bf07550b3fd4bfcb611a5c9b2eafc33', 'request_id': 'req-a68347ec-126b-46bd-be76-b4e2b9fbcb33'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:25.850274 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-a68347ec-126b-46bd-be76-b4e2b9fbcb33 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6bf07550b3fd4bfcb611a5c9b2eafc33', 'request_id': 'req-a68347ec-126b-46bd-be76-b4e2b9fbcb33'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:25.859211 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-a68347ec-126b-46bd-be76-b4e2b9fbcb33 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6bf07550b3fd4bfcb611a5c9b2eafc33', 'request_id': 'req-a68347ec-126b-46bd-be76-b4e2b9fbcb33'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:25.871256 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-a68347ec-126b-46bd-be76-b4e2b9fbcb33 admin admin] The token cache is being invalidate because project 6bf07550b3fd4bfcb611a5c9b2eafc33 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86828) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 10:48:25.871555 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-a68347ec-126b-46bd-be76-b4e2b9fbcb33 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:25.872220 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-a68347ec-126b-46bd-be76-b4e2b9fbcb33 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:25.873893 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 883/1795] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Jul 27 10:48:25 2022] DELETE /identity/v3/projects/6bf07550b3fd4bfcb611a5c9b2eafc33 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:48:25.877640 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68347ec-126b-46bd-be76-b4e2b9fbcb33 admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:25.877640 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-a68347ec-126b-46bd-be76-b4e2b9fbcb33 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:25.881838 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ea6f08-6ae8-448c-b182-14db7a3facd0 admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:25.917555 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:48:25.946102 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68347ec-126b-46bd-be76-b4e2b9fbcb33 admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:25.951320 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d240777-6bbc-41a5-8856-08b9269f33f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:25.951963 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d240777-6bbc-41a5-8856-08b9269f33f9 admin admin] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:25.952252 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d240777-6bbc-41a5-8856-08b9269f33f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:25.952523 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d240777-6bbc-41a5-8856-08b9269f33f9 admin admin] PATH_INFO: `/v3/projects` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:25.953239 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d240777-6bbc-41a5-8856-08b9269f33f9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:25.954414 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76eae2aa-48b0-417e-9740-b9e87c2238f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:25.955106 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76eae2aa-48b0-417e-9740-b9e87c2238f4 admin admin] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:25.955205 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d240777-6bbc-41a5-8856-08b9269f33f9 admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:25.955390 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76eae2aa-48b0-417e-9740-b9e87c2238f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:25.955663 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76eae2aa-48b0-417e-9740-b9e87c2238f4 admin admin] PATH_INFO: `/v3/projects` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:25.956328 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76eae2aa-48b0-417e-9740-b9e87c2238f4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:25.958236 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76eae2aa-48b0-417e-9740-b9e87c2238f4 admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:25.983410 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 913/1796] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Jul 27 10:48:25 2022] POST /identity/v3/projects => generated 367 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 10:48:25.984231 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d240777-6bbc-41a5-8856-08b9269f33f9 admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:25.985933 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-2d240777-6bbc-41a5-8856-08b9269f33f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:25.987273 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 884/1797] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Jul 27 10:48:25 2022] POST /identity/v3/projects => generated 387 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 10:48:25.988220 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76eae2aa-48b0-417e-9740-b9e87c2238f4 admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:25.991304 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-76eae2aa-48b0-417e-9740-b9e87c2238f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:26.050576 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d240777-6bbc-41a5-8856-08b9269f33f9 admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:26.054848 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76eae2aa-48b0-417e-9740-b9e87c2238f4 admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:26.055706 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d344eeb9-3e2c-4ba8-bf37-f86760b9072d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:26.056327 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d344eeb9-3e2c-4ba8-bf37-f86760b9072d admin admin] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:26.056611 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d344eeb9-3e2c-4ba8-bf37-f86760b9072d admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:26.056886 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d344eeb9-3e2c-4ba8-bf37-f86760b9072d admin admin] PATH_INFO: `/v3/users` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:26.057585 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d344eeb9-3e2c-4ba8-bf37-f86760b9072d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:26.059538 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d344eeb9-3e2c-4ba8-bf37-f86760b9072d admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:26.060519 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f701575-befb-4301-8f65-043d9064b3ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:26.061294 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f701575-befb-4301-8f65-043d9064b3ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:26.061588 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f701575-befb-4301-8f65-043d9064b3ab admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:26.061869 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f701575-befb-4301-8f65-043d9064b3ab admin admin] PATH_INFO: `/v3/users/4efb00e883f2427eaf9e2acf4752b0f5` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:26.063153 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f701575-befb-4301-8f65-043d9064b3ab admin admin] RBAC: Authorizing `identity:delete_user(user_id=4efb00e883f2427eaf9e2acf4752b0f5)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:26.065080 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f701575-befb-4301-8f65-043d9064b3ab admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:26.085015 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 914/1798] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Jul 27 10:48:25 2022] POST /identity/v3/users => generated 290 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 10:48:26.085015 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d344eeb9-3e2c-4ba8-bf37-f86760b9072d admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:26.085015 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-d344eeb9-3e2c-4ba8-bf37-f86760b9072d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:26.092288 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:48:26.098046 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d344eeb9-3e2c-4ba8-bf37-f86760b9072d admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:26.104871 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f36bd4-e889-4834-a512-12170d06d770 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:26.105571 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f36bd4-e889-4834-a512-12170d06d770 admin admin] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:26.105884 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f36bd4-e889-4834-a512-12170d06d770 admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:26.106209 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f36bd4-e889-4834-a512-12170d06d770 admin admin] PATH_INFO: `/v3/users` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:26.106915 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f36bd4-e889-4834-a512-12170d06d770 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:26.108840 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f36bd4-e889-4834-a512-12170d06d770 admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:26.118959 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-9f701575-befb-4301-8f65-043d9064b3ab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4efb00e883f2427eaf9e2acf4752b0f5'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:26.127820 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-9f701575-befb-4301-8f65-043d9064b3ab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4efb00e883f2427eaf9e2acf4752b0f5'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:26.130485 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 915/1799] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Jul 27 10:48:26 2022] POST /identity/v3/users => generated 284 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 10:48:26.131540 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f36bd4-e889-4834-a512-12170d06d770 admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:26.139094 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-9f701575-befb-4301-8f65-043d9064b3ab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4efb00e883f2427eaf9e2acf4752b0f5'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:26.139723 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-9f701575-befb-4301-8f65-043d9064b3ab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4efb00e883f2427eaf9e2acf4752b0f5'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:26.140436 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-9f701575-befb-4301-8f65-043d9064b3ab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4efb00e883f2427eaf9e2acf4752b0f5'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:26.143033 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f36bd4-e889-4834-a512-12170d06d770 admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:26.147647 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d3808b-3a3c-4fb5-8603-b4f0074e977f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:26.148253 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d3808b-3a3c-4fb5-8603-b4f0074e977f admin admin] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:26.148532 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d3808b-3a3c-4fb5-8603-b4f0074e977f admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:26.148814 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d3808b-3a3c-4fb5-8603-b4f0074e977f admin admin] PATH_INFO: `/v3/users` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:26.149475 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d3808b-3a3c-4fb5-8603-b4f0074e977f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:26.151416 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d3808b-3a3c-4fb5-8603-b4f0074e977f admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:26.162264 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 885/1800] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Jul 27 10:48:25 2022] DELETE /identity/v3/users/4efb00e883f2427eaf9e2acf4752b0f5 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:48:26.162264 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f701575-befb-4301-8f65-043d9064b3ab admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:26.163663 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-9f701575-befb-4301-8f65-043d9064b3ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:26.179507 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 916/1801] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Jul 27 10:48:26 2022] POST /identity/v3/users => generated 294 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 10:48:26.179507 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d3808b-3a3c-4fb5-8603-b4f0074e977f admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:26.179507 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-76d3808b-3a3c-4fb5-8603-b4f0074e977f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:26.201789 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d3808b-3a3c-4fb5-8603-b4f0074e977f admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:26.217626 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:48:26.249819 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f701575-befb-4301-8f65-043d9064b3ab admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:26.256487 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5a3039-2905-4dac-bbea-1ec10afef81c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:26.257324 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5a3039-2905-4dac-bbea-1ec10afef81c admin admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:26.257494 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5a3039-2905-4dac-bbea-1ec10afef81c admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:26.257779 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5a3039-2905-4dac-bbea-1ec10afef81c admin admin] PATH_INFO: `/v3/roles` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:26.258560 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5a3039-2905-4dac-bbea-1ec10afef81c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:26.260298 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5a3039-2905-4dac-bbea-1ec10afef81c admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:26.274465 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 886/1802] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 10:48:26 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 10:48:26.274465 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5a3039-2905-4dac-bbea-1ec10afef81c admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:26.278964 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-3e5a3039-2905-4dac-bbea-1ec10afef81c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:26.306701 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5a3039-2905-4dac-bbea-1ec10afef81c admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:26.307298 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7db29b-7e67-4bb3-a0a0-53aa87f38b3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:26.307943 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7db29b-7e67-4bb3-a0a0-53aa87f38b3d admin admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:26.308332 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7db29b-7e67-4bb3-a0a0-53aa87f38b3d admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:26.308620 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7db29b-7e67-4bb3-a0a0-53aa87f38b3d admin admin] PATH_INFO: `/v3/roles` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:26.310050 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7db29b-7e67-4bb3-a0a0-53aa87f38b3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:26.312083 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7db29b-7e67-4bb3-a0a0-53aa87f38b3d admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:26.320211 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262e0256-d8cb-46d0-99be-d266e8ec82a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:26.320905 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262e0256-d8cb-46d0-99be-d266e8ec82a8 admin admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:26.321187 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262e0256-d8cb-46d0-99be-d266e8ec82a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:26.321462 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262e0256-d8cb-46d0-99be-d266e8ec82a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:26.322164 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262e0256-d8cb-46d0-99be-d266e8ec82a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:26.323803 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262e0256-d8cb-46d0-99be-d266e8ec82a8 admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:26.331213 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 917/1803] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 10:48:26 2022] GET /identity/v3/roles => generated 1420 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 10:48:26.339490 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7db29b-7e67-4bb3-a0a0-53aa87f38b3d admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:26.354982 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 887/1804] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 10:48:26 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 10:48:26.356070 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262e0256-d8cb-46d0-99be-d266e8ec82a8 admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:26.370941 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7db29b-7e67-4bb3-a0a0-53aa87f38b3d admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:26.380533 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262e0256-d8cb-46d0-99be-d266e8ec82a8 admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:26.382815 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ff328a-f741-451c-bba0-5e2edc178dd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:26.383472 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ff328a-f741-451c-bba0-5e2edc178dd9 admin admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:26.383755 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ff328a-f741-451c-bba0-5e2edc178dd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:26.384056 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ff328a-f741-451c-bba0-5e2edc178dd9 admin admin] PATH_INFO: `/v3/roles` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:26.384782 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ff328a-f741-451c-bba0-5e2edc178dd9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:26.388588 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ff328a-f741-451c-bba0-5e2edc178dd9 admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:26.391037 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19a1b26-e987-497d-988b-a75699311e5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:26.391686 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19a1b26-e987-497d-988b-a75699311e5c admin admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:26.391945 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19a1b26-e987-497d-988b-a75699311e5c admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:26.392220 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19a1b26-e987-497d-988b-a75699311e5c admin admin] PATH_INFO: `/v3/roles` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:26.392914 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19a1b26-e987-497d-988b-a75699311e5c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:26.395461 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19a1b26-e987-497d-988b-a75699311e5c admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:26.410213 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 888/1805] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 10:48:26 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 10:48:26.410213 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19a1b26-e987-497d-988b-a75699311e5c admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:26.416807 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 918/1806] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 10:48:26 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 10:48:26.418203 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ff328a-f741-451c-bba0-5e2edc178dd9 admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:26.431150 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19a1b26-e987-497d-988b-a75699311e5c admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:26.440884 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ff328a-f741-451c-bba0-5e2edc178dd9 admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:26.442229 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b83573-94c8-4d5a-98b2-3a5a38e620c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:26.442983 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b83573-94c8-4d5a-98b2-3a5a38e620c2 admin admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:26.443285 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b83573-94c8-4d5a-98b2-3a5a38e620c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:26.443559 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b83573-94c8-4d5a-98b2-3a5a38e620c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:26.444255 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b83573-94c8-4d5a-98b2-3a5a38e620c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:26.446067 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b83573-94c8-4d5a-98b2-3a5a38e620c2 admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:26.470121 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516f61cd-34f5-441d-a659-9701d1412d99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:26.470854 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516f61cd-34f5-441d-a659-9701d1412d99 admin admin] REQUEST_METHOD: `PUT` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:26.471138 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516f61cd-34f5-441d-a659-9701d1412d99 admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:26.471413 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516f61cd-34f5-441d-a659-9701d1412d99 admin admin] PATH_INFO: `/v3/projects/0e1feac13d044638956a0d30726adddd/users/3efff22ca50741bc9e4d2b1f426f3d7f/roles/4c9fc0b3baed441da2126be41cc6b04a` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:26.474846 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 889/1807] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 10:48:26 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 10:48:26.475814 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b83573-94c8-4d5a-98b2-3a5a38e620c2 admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:26.488429 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b83573-94c8-4d5a-98b2-3a5a38e620c2 admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:26.494346 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7b8d93-525e-417b-92c7-23b53dd69e1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:26.495145 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7b8d93-525e-417b-92c7-23b53dd69e1a admin admin] REQUEST_METHOD: `PUT` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:26.495505 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7b8d93-525e-417b-92c7-23b53dd69e1a admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:26.495857 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7b8d93-525e-417b-92c7-23b53dd69e1a admin admin] PATH_INFO: `/v3/projects/ca74ae0b9f354340b1e8f54eca2450c4/users/10087ffcfc514c3e8400ff5c91ce5ceb/roles/4c9fc0b3baed441da2126be41cc6b04a` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:26.504361 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516f61cd-34f5-441d-a659-9701d1412d99 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0e1feac13d044638956a0d30726adddd, user_id=3efff22ca50741bc9e4d2b1f426f3d7f, role_id=4c9fc0b3baed441da2126be41cc6b04a)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:26.513530 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516f61cd-34f5-441d-a659-9701d1412d99 admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:26.518982 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7b8d93-525e-417b-92c7-23b53dd69e1a admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca74ae0b9f354340b1e8f54eca2450c4, user_id=10087ffcfc514c3e8400ff5c91ce5ceb, role_id=4c9fc0b3baed441da2126be41cc6b04a)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:26.523062 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7b8d93-525e-417b-92c7-23b53dd69e1a admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:26.550444 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 890/1808] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 10:48:26 2022] PUT /identity/v3/projects/ca74ae0b9f354340b1e8f54eca2450c4/users/10087ffcfc514c3e8400ff5c91ce5ceb/roles/4c9fc0b3baed441da2126be41cc6b04a => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:48:26.551384 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7b8d93-525e-417b-92c7-23b53dd69e1a admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:26.565763 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 919/1809] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 10:48:26 2022] PUT /identity/v3/projects/0e1feac13d044638956a0d30726adddd/users/3efff22ca50741bc9e4d2b1f426f3d7f/roles/4c9fc0b3baed441da2126be41cc6b04a => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:48:26.574639 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f75f0d8-f90f-4ae2-af45-d8145b26f613 None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:26.575282 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f75f0d8-f90f-4ae2-af45-d8145b26f613 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:26.575822 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f75f0d8-f90f-4ae2-af45-d8145b26f613 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:26.685801 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7b8d93-525e-417b-92c7-23b53dd69e1a admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:26.729138 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-4f75f0d8-f90f-4ae2-af45-d8145b26f613 None None] MFA Rules not processed for user `10087ffcfc514c3e8400ff5c91ce5ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:48:26.732048 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-4f75f0d8-f90f-4ae2-af45-d8145b26f613 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:26.781812 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7687160e-bc91-4307-b4d7-96364a23ab3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:26.782505 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7687160e-bc91-4307-b4d7-96364a23ab3a admin admin] REQUEST_METHOD: `PUT` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:26.782829 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7687160e-bc91-4307-b4d7-96364a23ab3a admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:26.783116 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7687160e-bc91-4307-b4d7-96364a23ab3a admin admin] PATH_INFO: `/v3/projects/13f7a28998894dbbb487b4d71426e8b3/users/0d7fac304b2447c3ae288a68e7e55e24/roles/4c9fc0b3baed441da2126be41cc6b04a` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:26.802161 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7687160e-bc91-4307-b4d7-96364a23ab3a admin admin] RBAC: Authorizing `identity:create_grant(project_id=13f7a28998894dbbb487b4d71426e8b3, user_id=0d7fac304b2447c3ae288a68e7e55e24, role_id=4c9fc0b3baed441da2126be41cc6b04a)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:26.805375 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7687160e-bc91-4307-b4d7-96364a23ab3a admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:26.846202 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 891/1810] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 10:48:26 2022] PUT /identity/v3/projects/13f7a28998894dbbb487b4d71426e8b3/users/0d7fac304b2447c3ae288a68e7e55e24/roles/4c9fc0b3baed441da2126be41cc6b04a => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:48:26.847590 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9b96d5-dc6f-4fd6-8dc2-a7665ea4ee6a None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:26.847890 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9b96d5-dc6f-4fd6-8dc2-a7665ea4ee6a None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:26.848175 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9b96d5-dc6f-4fd6-8dc2-a7665ea4ee6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:26.878300 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:48:26.891665 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-0c9b96d5-dc6f-4fd6-8dc2-a7665ea4ee6a None None] MFA Rules not processed for user `3efff22ca50741bc9e4d2b1f426f3d7f`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:48:26.893313 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-0c9b96d5-dc6f-4fd6-8dc2-a7665ea4ee6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:27.021328 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 892/1811] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:48:26 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:48:27.022198 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9b96d5-dc6f-4fd6-8dc2-a7665ea4ee6a None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:27.023875 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-0c9b96d5-dc6f-4fd6-8dc2-a7665ea4ee6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:27.064076 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 920/1812] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:48:26 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 497 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:48:27.065376 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d04ebb3-3d15-4e0d-9e38-6494fe973d1f None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:27.065671 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d04ebb3-3d15-4e0d-9e38-6494fe973d1f None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:27.065947 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d04ebb3-3d15-4e0d-9e38-6494fe973d1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:27.126122 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-6d04ebb3-3d15-4e0d-9e38-6494fe973d1f None None] MFA Rules not processed for user `0d7fac304b2447c3ae288a68e7e55e24`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:48:27.127787 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-6d04ebb3-3d15-4e0d-9e38-6494fe973d1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:27.133194 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:48:27.137435 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9b96d5-dc6f-4fd6-8dc2-a7665ea4ee6a None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:27.148043 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cd0d0b-a8f3-4d09-bfe6-6cffd9ecbad7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:27.149039 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cd0d0b-a8f3-4d09-bfe6-6cffd9ecbad7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:27.149522 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cd0d0b-a8f3-4d09-bfe6-6cffd9ecbad7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:27.149993 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cd0d0b-a8f3-4d09-bfe6-6cffd9ecbad7 admin admin] PATH_INFO: `/v3/projects/1b333cc578d44a2cb514d27097b3c359` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:27.152265 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cd0d0b-a8f3-4d09-bfe6-6cffd9ecbad7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1b333cc578d44a2cb514d27097b3c359)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:27.156727 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cd0d0b-a8f3-4d09-bfe6-6cffd9ecbad7 admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:27.222644 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 921/1813] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:48:27 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:48:27.222824 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-96cd0d0b-a8f3-4d09-bfe6-6cffd9ecbad7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1b333cc578d44a2cb514d27097b3c359', 'request_id': 'req-96cd0d0b-a8f3-4d09-bfe6-6cffd9ecbad7'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:27.222948 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d04ebb3-3d15-4e0d-9e38-6494fe973d1f None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:27.228809 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-96cd0d0b-a8f3-4d09-bfe6-6cffd9ecbad7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1b333cc578d44a2cb514d27097b3c359', 'request_id': 'req-96cd0d0b-a8f3-4d09-bfe6-6cffd9ecbad7'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:27.235795 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d04ebb3-3d15-4e0d-9e38-6494fe973d1f None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:27.237073 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-96cd0d0b-a8f3-4d09-bfe6-6cffd9ecbad7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1b333cc578d44a2cb514d27097b3c359', 'request_id': 'req-96cd0d0b-a8f3-4d09-bfe6-6cffd9ecbad7'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:27.247874 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-96cd0d0b-a8f3-4d09-bfe6-6cffd9ecbad7 admin admin] The token cache is being invalidate because project 1b333cc578d44a2cb514d27097b3c359 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86828) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 10:48:27.248164 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-96cd0d0b-a8f3-4d09-bfe6-6cffd9ecbad7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:27.248797 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-96cd0d0b-a8f3-4d09-bfe6-6cffd9ecbad7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:27.250557 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 893/1814] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Jul 27 10:48:27 2022] DELETE /identity/v3/projects/1b333cc578d44a2cb514d27097b3c359 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:48:27.251582 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cd0d0b-a8f3-4d09-bfe6-6cffd9ecbad7 admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:27.253326 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-96cd0d0b-a8f3-4d09-bfe6-6cffd9ecbad7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:27.289562 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:48:27.300430 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e164d6-2a1c-43cd-be5b-618395836e83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:27.301065 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e164d6-2a1c-43cd-be5b-618395836e83 admin admin] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:27.301351 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e164d6-2a1c-43cd-be5b-618395836e83 admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:27.301629 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e164d6-2a1c-43cd-be5b-618395836e83 admin admin] PATH_INFO: `/v3/projects` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:27.302307 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e164d6-2a1c-43cd-be5b-618395836e83 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:27.304292 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e164d6-2a1c-43cd-be5b-618395836e83 admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:27.327203 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cd0d0b-a8f3-4d09-bfe6-6cffd9ecbad7 admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:27.331811 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 922/1815] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Jul 27 10:48:27 2022] POST /identity/v3/projects => generated 367 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 10:48:27.332026 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc79875-b4a7-4d21-9f38-c6fef10f2b70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:27.332609 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc79875-b4a7-4d21-9f38-c6fef10f2b70 admin admin] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:27.332719 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e164d6-2a1c-43cd-be5b-618395836e83 admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:27.332957 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc79875-b4a7-4d21-9f38-c6fef10f2b70 admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:27.333160 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc79875-b4a7-4d21-9f38-c6fef10f2b70 admin admin] PATH_INFO: `/v3/projects` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:27.333932 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc79875-b4a7-4d21-9f38-c6fef10f2b70 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:27.334824 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-11e164d6-2a1c-43cd-be5b-618395836e83 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:27.336030 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc79875-b4a7-4d21-9f38-c6fef10f2b70 admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:27.363739 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 894/1816] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Jul 27 10:48:27 2022] POST /identity/v3/projects => generated 379 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 10:48:27.365007 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a66a4b2-ecad-4191-8c68-43ac7d7299ca None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:27.365274 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a66a4b2-ecad-4191-8c68-43ac7d7299ca None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:27.365551 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a66a4b2-ecad-4191-8c68-43ac7d7299ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:27.385009 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-1a66a4b2-ecad-4191-8c68-43ac7d7299ca None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:48:27.386203 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-1a66a4b2-ecad-4191-8c68-43ac7d7299ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:27.401588 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e164d6-2a1c-43cd-be5b-618395836e83 admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:27.440454 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:48:27.453087 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 895/1817] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:48:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:48:27.453949 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a66a4b2-ecad-4191-8c68-43ac7d7299ca None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:27.455345 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-1a66a4b2-ecad-4191-8c68-43ac7d7299ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:27.467789 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a66a4b2-ecad-4191-8c68-43ac7d7299ca None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:27.469573 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2869d438-e19c-4b1c-ab6d-bb073dad3200 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:27.470215 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2869d438-e19c-4b1c-ab6d-bb073dad3200 admin admin] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:27.470502 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2869d438-e19c-4b1c-ab6d-bb073dad3200 admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:27.470815 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2869d438-e19c-4b1c-ab6d-bb073dad3200 admin admin] PATH_INFO: `/v3/projects` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:27.471485 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2869d438-e19c-4b1c-ab6d-bb073dad3200 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:27.472546 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2b00ec-2813-4a9e-b751-5abae24a5750 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:27.473161 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2b00ec-2813-4a9e-b751-5abae24a5750 admin admin] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:27.473410 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2869d438-e19c-4b1c-ab6d-bb073dad3200 admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:27.473510 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2b00ec-2813-4a9e-b751-5abae24a5750 admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:27.473708 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2b00ec-2813-4a9e-b751-5abae24a5750 admin admin] PATH_INFO: `/v3/users` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:27.474372 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2b00ec-2813-4a9e-b751-5abae24a5750 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:27.476410 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2b00ec-2813-4a9e-b751-5abae24a5750 admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:27.500427 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 896/1818] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Jul 27 10:48:27 2022] POST /identity/v3/users => generated 290 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 10:48:27.502042 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2b00ec-2813-4a9e-b751-5abae24a5750 admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:27.503041 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 923/1819] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Jul 27 10:48:27 2022] POST /identity/v3/projects => generated 385 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 10:48:27.504456 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258ac830-5bf8-4fc2-91fe-0f9570a1b22a None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:27.504815 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258ac830-5bf8-4fc2-91fe-0f9570a1b22a None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:27.505185 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258ac830-5bf8-4fc2-91fe-0f9570a1b22a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:27.519045 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2b00ec-2813-4a9e-b751-5abae24a5750 admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:27.525913 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-258ac830-5bf8-4fc2-91fe-0f9570a1b22a None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:48:27.527146 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-258ac830-5bf8-4fc2-91fe-0f9570a1b22a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:27.581743 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e0cdb3-9775-444d-816b-8b299bcbc1c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:27.582369 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e0cdb3-9775-444d-816b-8b299bcbc1c5 admin admin] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:27.582693 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e0cdb3-9775-444d-816b-8b299bcbc1c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:27.582977 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e0cdb3-9775-444d-816b-8b299bcbc1c5 admin admin] PATH_INFO: `/v3/users` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:27.583640 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e0cdb3-9775-444d-816b-8b299bcbc1c5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:27.585609 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e0cdb3-9775-444d-816b-8b299bcbc1c5 admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:27.588248 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 924/1820] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:48:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:48:27.589096 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258ac830-5bf8-4fc2-91fe-0f9570a1b22a None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:27.599806 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258ac830-5bf8-4fc2-91fe-0f9570a1b22a None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:27.604739 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b618b84d-f8c6-49e3-929e-2438baa4c4a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:27.605353 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b618b84d-f8c6-49e3-929e-2438baa4c4a2 admin admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:27.605669 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b618b84d-f8c6-49e3-929e-2438baa4c4a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:27.605943 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b618b84d-f8c6-49e3-929e-2438baa4c4a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:27.606683 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b618b84d-f8c6-49e3-929e-2438baa4c4a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:27.608462 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b618b84d-f8c6-49e3-929e-2438baa4c4a2 admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:27.608786 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 897/1821] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Jul 27 10:48:27 2022] POST /identity/v3/users => generated 290 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 10:48:27.609633 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e0cdb3-9775-444d-816b-8b299bcbc1c5 admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:27.618935 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 925/1822] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 10:48:27 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 10:48:27.619632 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b618b84d-f8c6-49e3-929e-2438baa4c4a2 admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:27.620629 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e0cdb3-9775-444d-816b-8b299bcbc1c5 admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:27.626057 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bbffb1-70a2-4d5d-9fef-8a09113798b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:27.626970 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bbffb1-70a2-4d5d-9fef-8a09113798b9 admin admin] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:27.627257 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bbffb1-70a2-4d5d-9fef-8a09113798b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:27.627533 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bbffb1-70a2-4d5d-9fef-8a09113798b9 admin admin] PATH_INFO: `/v3/users` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:27.628185 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43bbffb1-70a2-4d5d-9fef-8a09113798b9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:27.630092 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43bbffb1-70a2-4d5d-9fef-8a09113798b9 admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:27.631742 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b618b84d-f8c6-49e3-929e-2438baa4c4a2 admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:27.636345 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a920efe-5053-47c0-ae3b-6eddf2718a3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:27.637040 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a920efe-5053-47c0-ae3b-6eddf2718a3e admin admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:27.637792 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a920efe-5053-47c0-ae3b-6eddf2718a3e admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:27.638075 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a920efe-5053-47c0-ae3b-6eddf2718a3e admin admin] PATH_INFO: `/v3/domains` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:27.638837 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a920efe-5053-47c0-ae3b-6eddf2718a3e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86827) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:48:27.639143 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a920efe-5053-47c0-ae3b-6eddf2718a3e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:27.640705 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a920efe-5053-47c0-ae3b-6eddf2718a3e admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:27.652742 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 898/1823] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Jul 27 10:48:27 2022] POST /identity/v3/users => generated 299 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 10:48:27.653563 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bbffb1-70a2-4d5d-9fef-8a09113798b9 admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:27.655351 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 926/1824] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Jul 27 10:48:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:48:27.656430 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a920efe-5053-47c0-ae3b-6eddf2718a3e admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:27.668391 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bbffb1-70a2-4d5d-9fef-8a09113798b9 admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:27.668764 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a920efe-5053-47c0-ae3b-6eddf2718a3e admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:27.673241 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b635a55f-4e07-4509-86f1-09b45aaeb9bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:27.673521 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadeb7b2-c13f-49bc-9c29-bf1b3c4e062b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:27.673936 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b635a55f-4e07-4509-86f1-09b45aaeb9bb admin admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:27.674228 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b635a55f-4e07-4509-86f1-09b45aaeb9bb admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:27.674469 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadeb7b2-c13f-49bc-9c29-bf1b3c4e062b admin admin] REQUEST_METHOD: `PUT` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:27.674576 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b635a55f-4e07-4509-86f1-09b45aaeb9bb admin admin] PATH_INFO: `/v3/roles` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:27.674787 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadeb7b2-c13f-49bc-9c29-bf1b3c4e062b admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:27.675088 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadeb7b2-c13f-49bc-9c29-bf1b3c4e062b admin admin] PATH_INFO: `/v3/projects/3050f16606c44d039c13200dd1e3566b/users/adedde348d9540b6997d1108513c702d/roles/21156560eb6b436bb567a40b4976f3f4` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:27.675201 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b635a55f-4e07-4509-86f1-09b45aaeb9bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:27.676810 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b635a55f-4e07-4509-86f1-09b45aaeb9bb admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:27.679944 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=90,keystone:INSERT=10 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:48:27.692729 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 899/1825] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 10:48:27 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 10:48:27.692913 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadeb7b2-c13f-49bc-9c29-bf1b3c4e062b admin admin] RBAC: Authorizing `identity:create_grant(project_id=3050f16606c44d039c13200dd1e3566b, user_id=adedde348d9540b6997d1108513c702d, role_id=21156560eb6b436bb567a40b4976f3f4)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:27.693661 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b635a55f-4e07-4509-86f1-09b45aaeb9bb admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:27.695790 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadeb7b2-c13f-49bc-9c29-bf1b3c4e062b admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:27.707253 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b635a55f-4e07-4509-86f1-09b45aaeb9bb admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:27.712152 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 927/1826] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 10:48:27 2022] PUT /identity/v3/projects/3050f16606c44d039c13200dd1e3566b/users/adedde348d9540b6997d1108513c702d/roles/21156560eb6b436bb567a40b4976f3f4 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:48:27.713121 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadeb7b2-c13f-49bc-9c29-bf1b3c4e062b admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:27.793029 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2160d1-49be-457c-b006-5d9fda203065 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:27.793721 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2160d1-49be-457c-b006-5d9fda203065 admin admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:27.793995 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2160d1-49be-457c-b006-5d9fda203065 admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:27.794276 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2160d1-49be-457c-b006-5d9fda203065 admin admin] PATH_INFO: `/v3/roles` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:27.795141 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2160d1-49be-457c-b006-5d9fda203065 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:27.796768 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2160d1-49be-457c-b006-5d9fda203065 admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:27.806044 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadeb7b2-c13f-49bc-9c29-bf1b3c4e062b admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:27.807189 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 900/1827] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 10:48:27 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 10:48:27.811688 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2160d1-49be-457c-b006-5d9fda203065 admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:27.814866 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f312ba2b-3abb-4fdc-83d2-8fafe3f269be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:27.815742 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f312ba2b-3abb-4fdc-83d2-8fafe3f269be admin admin] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:27.816215 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f312ba2b-3abb-4fdc-83d2-8fafe3f269be admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:27.816860 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f312ba2b-3abb-4fdc-83d2-8fafe3f269be admin admin] PATH_INFO: `/v3/projects` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:27.818307 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f312ba2b-3abb-4fdc-83d2-8fafe3f269be admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:27.820757 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f312ba2b-3abb-4fdc-83d2-8fafe3f269be admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:27.821564 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2160d1-49be-457c-b006-5d9fda203065 admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:27.827153 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34089ec-854b-45d0-8fe8-babe7ec8f003 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:27.828153 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34089ec-854b-45d0-8fe8-babe7ec8f003 admin admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:27.828443 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34089ec-854b-45d0-8fe8-babe7ec8f003 admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:27.828724 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34089ec-854b-45d0-8fe8-babe7ec8f003 admin admin] PATH_INFO: `/v3/roles` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:27.829427 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34089ec-854b-45d0-8fe8-babe7ec8f003 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:27.831065 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34089ec-854b-45d0-8fe8-babe7ec8f003 admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:27.843765 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 901/1828] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 10:48:27 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 10:48:27.847887 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34089ec-854b-45d0-8fe8-babe7ec8f003 admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:27.856496 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 928/1829] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Jul 27 10:48:27 2022] POST /identity/v3/projects => generated 409 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 10:48:27.858460 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f312ba2b-3abb-4fdc-83d2-8fafe3f269be admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:27.864079 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:48:27.864608 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34089ec-854b-45d0-8fe8-babe7ec8f003 admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:27.958291 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f312ba2b-3abb-4fdc-83d2-8fafe3f269be admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:27.960741 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcadcdd3-d3c9-491a-96aa-1b0560524ed4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:27.961563 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcadcdd3-d3c9-491a-96aa-1b0560524ed4 admin admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:27.961976 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcadcdd3-d3c9-491a-96aa-1b0560524ed4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:27.962262 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcadcdd3-d3c9-491a-96aa-1b0560524ed4 admin admin] PATH_INFO: `/v3/roles` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:27.962999 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcadcdd3-d3c9-491a-96aa-1b0560524ed4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:27.964156 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5128e3-3e80-43b0-9e0b-1eeb0abc634f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:27.964645 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcadcdd3-d3c9-491a-96aa-1b0560524ed4 admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:27.964893 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5128e3-3e80-43b0-9e0b-1eeb0abc634f admin admin] REQUEST_METHOD: `PUT` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:27.965379 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5128e3-3e80-43b0-9e0b-1eeb0abc634f admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:27.965803 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5128e3-3e80-43b0-9e0b-1eeb0abc634f admin admin] PATH_INFO: `/v3/projects/eff6b0631db74f47816a6669bc1e0368/users/bc07a6fadb4944e19e1b5f2b4380e289/roles/21156560eb6b436bb567a40b4976f3f4` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:27.975409 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 902/1830] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 10:48:27 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 10:48:27.976229 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcadcdd3-d3c9-491a-96aa-1b0560524ed4 admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:27.985814 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5128e3-3e80-43b0-9e0b-1eeb0abc634f admin admin] RBAC: Authorizing `identity:create_grant(project_id=eff6b0631db74f47816a6669bc1e0368, user_id=bc07a6fadb4944e19e1b5f2b4380e289, role_id=21156560eb6b436bb567a40b4976f3f4)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:27.986509 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcadcdd3-d3c9-491a-96aa-1b0560524ed4 admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:27.988566 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5128e3-3e80-43b0-9e0b-1eeb0abc634f admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:27.991112 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290d1a1f-b005-47b1-b83f-f73404230f7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:27.991895 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290d1a1f-b005-47b1-b83f-f73404230f7b admin admin] REQUEST_METHOD: `PUT` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:27.992187 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290d1a1f-b005-47b1-b83f-f73404230f7b admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:27.992468 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290d1a1f-b005-47b1-b83f-f73404230f7b admin admin] PATH_INFO: `/v3/projects/2499a2c444f94b91a8d2ac27daccf392/users/cf861046cc254a48b648f35b5d9e83e6/roles/4c9fc0b3baed441da2126be41cc6b04a` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:28.003220 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 929/1831] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 10:48:27 2022] PUT /identity/v3/projects/eff6b0631db74f47816a6669bc1e0368/users/bc07a6fadb4944e19e1b5f2b4380e289/roles/21156560eb6b436bb567a40b4976f3f4 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:48:28.004013 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5128e3-3e80-43b0-9e0b-1eeb0abc634f admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:28.010366 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290d1a1f-b005-47b1-b83f-f73404230f7b admin admin] RBAC: Authorizing `identity:create_grant(project_id=2499a2c444f94b91a8d2ac27daccf392, user_id=cf861046cc254a48b648f35b5d9e83e6, role_id=4c9fc0b3baed441da2126be41cc6b04a)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:28.013197 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290d1a1f-b005-47b1-b83f-f73404230f7b admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:28.026942 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 903/1832] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 10:48:27 2022] PUT /identity/v3/projects/2499a2c444f94b91a8d2ac27daccf392/users/cf861046cc254a48b648f35b5d9e83e6/roles/4c9fc0b3baed441da2126be41cc6b04a => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:48:28.027766 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290d1a1f-b005-47b1-b83f-f73404230f7b admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:28.069022 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5128e3-3e80-43b0-9e0b-1eeb0abc634f admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:28.086996 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:48:28.093679 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290d1a1f-b005-47b1-b83f-f73404230f7b admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:28.098827 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3177b26-f181-4141-a6cf-ea3e5a7d1f56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:28.099509 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3177b26-f181-4141-a6cf-ea3e5a7d1f56 admin admin] REQUEST_METHOD: `PUT` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:28.099795 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3177b26-f181-4141-a6cf-ea3e5a7d1f56 admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:28.100077 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3177b26-f181-4141-a6cf-ea3e5a7d1f56 admin admin] PATH_INFO: `/v3/projects/3050f16606c44d039c13200dd1e3566b/users/adedde348d9540b6997d1108513c702d/roles/21156560eb6b436bb567a40b4976f3f4` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:28.102109 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3177b26-f181-4141-a6cf-ea3e5a7d1f56 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3050f16606c44d039c13200dd1e3566b, user_id=adedde348d9540b6997d1108513c702d, role_id=21156560eb6b436bb567a40b4976f3f4)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:28.104794 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3177b26-f181-4141-a6cf-ea3e5a7d1f56 admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:28.115826 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 904/1833] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 10:48:28 2022] PUT /identity/v3/projects/3050f16606c44d039c13200dd1e3566b/users/adedde348d9540b6997d1108513c702d/roles/21156560eb6b436bb567a40b4976f3f4 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:48:28.116707 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3177b26-f181-4141-a6cf-ea3e5a7d1f56 admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:28.127401 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a077073-39c5-4252-93ba-5e92d4e70831 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:28.128034 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a077073-39c5-4252-93ba-5e92d4e70831 admin admin] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:28.128314 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a077073-39c5-4252-93ba-5e92d4e70831 admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:28.128598 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a077073-39c5-4252-93ba-5e92d4e70831 admin admin] PATH_INFO: `/v3/users` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:28.129263 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a077073-39c5-4252-93ba-5e92d4e70831 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:28.131230 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a077073-39c5-4252-93ba-5e92d4e70831 admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:28.156257 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 930/1834] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Jul 27 10:48:28 2022] POST /identity/v3/users => generated 305 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 10:48:28.156257 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329add1a-288b-404c-9dc2-b308f5ab50d9 None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:28.156257 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329add1a-288b-404c-9dc2-b308f5ab50d9 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:28.156257 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329add1a-288b-404c-9dc2-b308f5ab50d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:28.183761 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-329add1a-288b-404c-9dc2-b308f5ab50d9 None None] MFA Rules not processed for user `cf861046cc254a48b648f35b5d9e83e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:48:28.185817 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-329add1a-288b-404c-9dc2-b308f5ab50d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:28.191972 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3177b26-f181-4141-a6cf-ea3e5a7d1f56 admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:28.195672 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:48:28.196801 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e39ab3-e9fe-48d7-85f7-d49f924d6af1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:28.197436 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e39ab3-e9fe-48d7-85f7-d49f924d6af1 admin admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:28.197744 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e39ab3-e9fe-48d7-85f7-d49f924d6af1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:28.197978 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e39ab3-e9fe-48d7-85f7-d49f924d6af1 admin admin] PATH_INFO: `/v3/roles` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:28.198719 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e39ab3-e9fe-48d7-85f7-d49f924d6af1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:28.200882 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e39ab3-e9fe-48d7-85f7-d49f924d6af1 admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:28.214468 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 905/1835] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 10:48:28 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 10:48:28.214468 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a663cc-dc4e-4f0f-ab72-de8389ae6558 None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:28.214468 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a663cc-dc4e-4f0f-ab72-de8389ae6558 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:28.214468 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a663cc-dc4e-4f0f-ab72-de8389ae6558 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:28.249784 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-f5a663cc-dc4e-4f0f-ab72-de8389ae6558 None None] MFA Rules not processed for user `adedde348d9540b6997d1108513c702d`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:48:28.251064 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-f5a663cc-dc4e-4f0f-ab72-de8389ae6558 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:28.261293 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 931/1836] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:48:28 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:48:28.263224 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329add1a-288b-404c-9dc2-b308f5ab50d9 None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:28.276526 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329add1a-288b-404c-9dc2-b308f5ab50d9 None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:28.282433 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1fcceb-2a9b-427d-8a8e-3096a984a1e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:28.283311 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1fcceb-2a9b-427d-8a8e-3096a984a1e5 admin admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:28.283596 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1fcceb-2a9b-427d-8a8e-3096a984a1e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:28.283873 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1fcceb-2a9b-427d-8a8e-3096a984a1e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:28.284575 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1fcceb-2a9b-427d-8a8e-3096a984a1e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:28.286345 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1fcceb-2a9b-427d-8a8e-3096a984a1e5 admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:28.297245 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 932/1837] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 10:48:28 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 10:48:28.298068 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1fcceb-2a9b-427d-8a8e-3096a984a1e5 admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:28.308371 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1fcceb-2a9b-427d-8a8e-3096a984a1e5 admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:28.313239 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb91a18-ed0f-4c51-831b-dbde0b4923e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:28.313909 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb91a18-ed0f-4c51-831b-dbde0b4923e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:28.314191 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb91a18-ed0f-4c51-831b-dbde0b4923e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:28.314470 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb91a18-ed0f-4c51-831b-dbde0b4923e6 admin admin] PATH_INFO: `/v3/projects/eff6b0631db74f47816a6669bc1e0368/users/bc07a6fadb4944e19e1b5f2b4380e289/roles/21156560eb6b436bb567a40b4976f3f4` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:28.316612 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb91a18-ed0f-4c51-831b-dbde0b4923e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eff6b0631db74f47816a6669bc1e0368, user_id=bc07a6fadb4944e19e1b5f2b4380e289, role_id=21156560eb6b436bb567a40b4976f3f4)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:28.319291 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb91a18-ed0f-4c51-831b-dbde0b4923e6 admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:28.331452 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 933/1838] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 10:48:28 2022] PUT /identity/v3/projects/eff6b0631db74f47816a6669bc1e0368/users/bc07a6fadb4944e19e1b5f2b4380e289/roles/21156560eb6b436bb567a40b4976f3f4 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:48:28.331990 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb91a18-ed0f-4c51-831b-dbde0b4923e6 admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:28.348800 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 906/1839] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:48:28 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:48:28.357525 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a663cc-dc4e-4f0f-ab72-de8389ae6558 None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:28.370140 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a663cc-dc4e-4f0f-ab72-de8389ae6558 None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:28.400409 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb91a18-ed0f-4c51-831b-dbde0b4923e6 admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:28.405117 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb51a79b-7afd-4f9d-98d3-622419224f66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:28.405742 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb51a79b-7afd-4f9d-98d3-622419224f66 admin admin] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:28.406027 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb51a79b-7afd-4f9d-98d3-622419224f66 admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:28.406357 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb51a79b-7afd-4f9d-98d3-622419224f66 admin admin] PATH_INFO: `/v3/projects` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:28.406997 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb51a79b-7afd-4f9d-98d3-622419224f66 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:28.408970 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb51a79b-7afd-4f9d-98d3-622419224f66 admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:28.430260 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9acc6d-3e82-4dc8-8443-678ee22bf040 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:28.430931 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9acc6d-3e82-4dc8-8443-678ee22bf040 admin admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:28.431213 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9acc6d-3e82-4dc8-8443-678ee22bf040 admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:28.431486 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9acc6d-3e82-4dc8-8443-678ee22bf040 admin admin] PATH_INFO: `/v3/roles` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:28.432181 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9acc6d-3e82-4dc8-8443-678ee22bf040 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:28.434295 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9acc6d-3e82-4dc8-8443-678ee22bf040 admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:28.436790 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 934/1840] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Jul 27 10:48:28 2022] POST /identity/v3/projects => generated 377 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 10:48:28.438069 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32b2323-6ec1-436b-a3e4-c9d75e8f68ed None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:28.438362 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32b2323-6ec1-436b-a3e4-c9d75e8f68ed None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:28.438662 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32b2323-6ec1-436b-a3e4-c9d75e8f68ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:28.444519 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 907/1841] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 10:48:28 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 10:48:28.445449 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9acc6d-3e82-4dc8-8443-678ee22bf040 admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:28.447100 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-3e9acc6d-3e82-4dc8-8443-678ee22bf040 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:28.457143 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:48:28.470364 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-e32b2323-6ec1-436b-a3e4-c9d75e8f68ed None None] MFA Rules not processed for user `bc07a6fadb4944e19e1b5f2b4380e289`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:48:28.471629 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-e32b2323-6ec1-436b-a3e4-c9d75e8f68ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:28.506322 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9acc6d-3e82-4dc8-8443-678ee22bf040 admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:28.536793 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:48:28.542527 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 935/1842] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:48:28 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:48:28.543752 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32b2323-6ec1-436b-a3e4-c9d75e8f68ed None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:28.563893 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d10b4b-e9df-46b2-909c-2f3d9db4c57e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '864d35f268cf45148756109759bb2743', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:28.564524 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d10b4b-e9df-46b2-909c-2f3d9db4c57e service cinder] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:28.564813 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d10b4b-e9df-46b2-909c-2f3d9db4c57e service cinder] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:28.565088 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d10b4b-e9df-46b2-909c-2f3d9db4c57e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:28.607529 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32b2323-6ec1-436b-a3e4-c9d75e8f68ed None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:28.612250 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7890636-5d53-4689-a142-6efc8799f00c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:28.612883 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7890636-5d53-4689-a142-6efc8799f00c admin admin] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:28.613168 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7890636-5d53-4689-a142-6efc8799f00c admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:28.613448 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7890636-5d53-4689-a142-6efc8799f00c admin admin] PATH_INFO: `/v3/users` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:28.614113 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7890636-5d53-4689-a142-6efc8799f00c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:28.616101 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7890636-5d53-4689-a142-6efc8799f00c admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:28.625557 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d10b4b-e9df-46b2-909c-2f3d9db4c57e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:28.627289 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d10b4b-e9df-46b2-909c-2f3d9db4c57e service cinder] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:28.641450 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 936/1843] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Jul 27 10:48:28 2022] POST /identity/v3/users => generated 295 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 10:48:28.641634 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 908/1844] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:48:28 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 10:48:28.641735 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7890636-5d53-4689-a142-6efc8799f00c admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:28.641838 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d10b4b-e9df-46b2-909c-2f3d9db4c57e service cinder] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:28.641838 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-51d10b4b-e9df-46b2-909c-2f3d9db4c57e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:28.653271 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d10b4b-e9df-46b2-909c-2f3d9db4c57e service cinder] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:28.653570 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7890636-5d53-4689-a142-6efc8799f00c admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:28.657894 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99064c0-f839-4c78-99b2-d88226049bbd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '864d35f268cf45148756109759bb2743', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:28.658753 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99064c0-f839-4c78-99b2-d88226049bbd service cinder] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:28.659075 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99064c0-f839-4c78-99b2-d88226049bbd service cinder] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:28.659558 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99064c0-f839-4c78-99b2-d88226049bbd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:28.660220 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25363a13-7441-479c-8957-009f699512bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:28.661206 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25363a13-7441-479c-8957-009f699512bb admin admin] REQUEST_METHOD: `PUT` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:28.661596 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25363a13-7441-479c-8957-009f699512bb admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:28.662182 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25363a13-7441-479c-8957-009f699512bb admin admin] PATH_INFO: `/v3/projects/f9dbc435a6354c1a8a3d16234c20b5db/users/fa647284289e40ecae75fb2b8c44b5cb/roles/4c9fc0b3baed441da2126be41cc6b04a` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:28.672418 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99064c0-f839-4c78-99b2-d88226049bbd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:28.673909 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99064c0-f839-4c78-99b2-d88226049bbd service cinder] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:28.681867 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25363a13-7441-479c-8957-009f699512bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=f9dbc435a6354c1a8a3d16234c20b5db, user_id=fa647284289e40ecae75fb2b8c44b5cb, role_id=4c9fc0b3baed441da2126be41cc6b04a)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:28.685221 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25363a13-7441-479c-8957-009f699512bb admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:28.686902 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 909/1845] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:48:28 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 10:48:28.687611 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99064c0-f839-4c78-99b2-d88226049bbd service cinder] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:28.702543 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 937/1846] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 10:48:28 2022] PUT /identity/v3/projects/f9dbc435a6354c1a8a3d16234c20b5db/users/fa647284289e40ecae75fb2b8c44b5cb/roles/4c9fc0b3baed441da2126be41cc6b04a => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:48:28.703790 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25363a13-7441-479c-8957-009f699512bb admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:28.704233 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99064c0-f839-4c78-99b2-d88226049bbd service cinder] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:28.769752 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25363a13-7441-479c-8957-009f699512bb admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:28.773659 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a11b825-09c3-48be-9d58-1799ccf9126e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '864d35f268cf45148756109759bb2743', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:28.774285 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a11b825-09c3-48be-9d58-1799ccf9126e service cinder] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:28.774573 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a11b825-09c3-48be-9d58-1799ccf9126e service cinder] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:28.774879 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a11b825-09c3-48be-9d58-1799ccf9126e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:28.776846 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-0a11b825-09c3-48be-9d58-1799ccf9126e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:28.782293 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b811881d-78e7-4c03-8d4b-84be450185ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:28.783134 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b811881d-78e7-4c03-8d4b-84be450185ac admin admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:28.783629 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b811881d-78e7-4c03-8d4b-84be450185ac admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:28.784012 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b811881d-78e7-4c03-8d4b-84be450185ac admin admin] PATH_INFO: `/v3/roles` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:28.784826 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b811881d-78e7-4c03-8d4b-84be450185ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:28.786823 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b811881d-78e7-4c03-8d4b-84be450185ac admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:28.788540 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:48:28.808981 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 910/1847] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 10:48:28 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 10:48:28.808981 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4a54c2-1add-40aa-b9f7-50a694adac54 None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:28.808981 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4a54c2-1add-40aa-b9f7-50a694adac54 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:28.808981 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4a54c2-1add-40aa-b9f7-50a694adac54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:28.835003 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-8e4a54c2-1add-40aa-b9f7-50a694adac54 None None] MFA Rules not processed for user `fa647284289e40ecae75fb2b8c44b5cb`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:48:28.836299 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-8e4a54c2-1add-40aa-b9f7-50a694adac54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:28.841835 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a11b825-09c3-48be-9d58-1799ccf9126e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:28.843309 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a11b825-09c3-48be-9d58-1799ccf9126e service cinder] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:28.852209 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 938/1848] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:48:28 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 10:48:28.852993 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a11b825-09c3-48be-9d58-1799ccf9126e service cinder] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:28.864347 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a11b825-09c3-48be-9d58-1799ccf9126e service cinder] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:28.870276 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6a1cac-9166-4328-b7d5-03530bab90f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:28.870888 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6a1cac-9166-4328-b7d5-03530bab90f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:28.871211 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6a1cac-9166-4328-b7d5-03530bab90f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:28.872036 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6a1cac-9166-4328-b7d5-03530bab90f8 admin admin] PATH_INFO: `/v3/users/0d7fac304b2447c3ae288a68e7e55e24` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:28.873238 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6a1cac-9166-4328-b7d5-03530bab90f8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0d7fac304b2447c3ae288a68e7e55e24)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:28.875214 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6a1cac-9166-4328-b7d5-03530bab90f8 admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:28.889444 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:48:28.911726 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 911/1849] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:48:28 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:48:28.912561 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4a54c2-1add-40aa-b9f7-50a694adac54 None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:28.925874 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4a54c2-1add-40aa-b9f7-50a694adac54 None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:28.931878 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e830c5db-abfb-4dce-9bd2-2828708e7e18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:28.932700 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e830c5db-abfb-4dce-9bd2-2828708e7e18 admin admin] REQUEST_METHOD: `PUT` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:28.932989 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e830c5db-abfb-4dce-9bd2-2828708e7e18 admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:28.933274 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e830c5db-abfb-4dce-9bd2-2828708e7e18 admin admin] PATH_INFO: `/v3/projects/4d21e17346b343419d55a98ae4b9728f/users/ebf4e0c178664cb6913ed3698ef5c7ce/roles/21156560eb6b436bb567a40b4976f3f4` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:28.949940 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-6c6a1cac-9166-4328-b7d5-03530bab90f8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0d7fac304b2447c3ae288a68e7e55e24'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:28.959093 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-6c6a1cac-9166-4328-b7d5-03530bab90f8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0d7fac304b2447c3ae288a68e7e55e24'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:28.960067 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e830c5db-abfb-4dce-9bd2-2828708e7e18 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4d21e17346b343419d55a98ae4b9728f, user_id=ebf4e0c178664cb6913ed3698ef5c7ce, role_id=21156560eb6b436bb567a40b4976f3f4)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:28.964113 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e830c5db-abfb-4dce-9bd2-2828708e7e18 admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:28.968454 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-6c6a1cac-9166-4328-b7d5-03530bab90f8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0d7fac304b2447c3ae288a68e7e55e24'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:28.969273 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-6c6a1cac-9166-4328-b7d5-03530bab90f8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0d7fac304b2447c3ae288a68e7e55e24'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:28.969936 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-6c6a1cac-9166-4328-b7d5-03530bab90f8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0d7fac304b2447c3ae288a68e7e55e24'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:28.983888 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 912/1850] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 10:48:28 2022] PUT /identity/v3/projects/4d21e17346b343419d55a98ae4b9728f/users/ebf4e0c178664cb6913ed3698ef5c7ce/roles/21156560eb6b436bb567a40b4976f3f4 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:48:28.984936 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e830c5db-abfb-4dce-9bd2-2828708e7e18 admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:28.986903 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-e830c5db-abfb-4dce-9bd2-2828708e7e18 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:29.010631 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 939/1851] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Jul 27 10:48:28 2022] DELETE /identity/v3/users/0d7fac304b2447c3ae288a68e7e55e24 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:48:29.010631 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6a1cac-9166-4328-b7d5-03530bab90f8 admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:29.013551 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-6c6a1cac-9166-4328-b7d5-03530bab90f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:29.097344 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e830c5db-abfb-4dce-9bd2-2828708e7e18 admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:29.129657 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6a1cac-9166-4328-b7d5-03530bab90f8 admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:29.141821 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8752f55a-f6ef-436b-8158-003c859a5e75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:29.142977 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8752f55a-f6ef-436b-8158-003c859a5e75 admin admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:29.143392 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8752f55a-f6ef-436b-8158-003c859a5e75 admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:29.143716 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8752f55a-f6ef-436b-8158-003c859a5e75 admin admin] PATH_INFO: `/v3/roles` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:29.144727 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8752f55a-f6ef-436b-8158-003c859a5e75 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:29.147930 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8752f55a-f6ef-436b-8158-003c859a5e75 admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:29.169877 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 940/1852] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 10:48:29 2022] GET /identity/v3/roles => generated 1420 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 10:48:29.171575 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945478a2-6e75-4721-ab23-bac9042e3551 None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:29.171869 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945478a2-6e75-4721-ab23-bac9042e3551 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:29.172152 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945478a2-6e75-4721-ab23-bac9042e3551 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:29.198625 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf374fdd-633f-4340-9f15-1b0f688c24ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:29.200928 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-945478a2-6e75-4721-ab23-bac9042e3551 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:48:29.201649 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf374fdd-633f-4340-9f15-1b0f688c24ef admin admin] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:29.201937 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf374fdd-633f-4340-9f15-1b0f688c24ef admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:29.202250 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf374fdd-633f-4340-9f15-1b0f688c24ef admin admin] PATH_INFO: `/v3/projects` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:29.203952 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-945478a2-6e75-4721-ab23-bac9042e3551 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:29.205739 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf374fdd-633f-4340-9f15-1b0f688c24ef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:29.208764 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf374fdd-633f-4340-9f15-1b0f688c24ef admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:29.219732 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 941/1853] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:48:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:48:29.220684 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945478a2-6e75-4721-ab23-bac9042e3551 None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:29.238920 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:48:29.247177 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945478a2-6e75-4721-ab23-bac9042e3551 None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:29.252803 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4beebb-5692-4eea-8880-814242dee0a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:29.253483 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4beebb-5692-4eea-8880-814242dee0a7 admin admin] REQUEST_METHOD: `PUT` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:29.253766 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4beebb-5692-4eea-8880-814242dee0a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:29.254043 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4beebb-5692-4eea-8880-814242dee0a7 admin admin] PATH_INFO: `/v3/projects/4d21e17346b343419d55a98ae4b9728f/users/ebf4e0c178664cb6913ed3698ef5c7ce/roles/21156560eb6b436bb567a40b4976f3f4` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:29.262189 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4beebb-5692-4eea-8880-814242dee0a7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4d21e17346b343419d55a98ae4b9728f, user_id=ebf4e0c178664cb6913ed3698ef5c7ce, role_id=21156560eb6b436bb567a40b4976f3f4)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:29.262413 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4beebb-5692-4eea-8880-814242dee0a7 admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:29.269461 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 913/1854] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Jul 27 10:48:28 2022] POST /identity/v3/projects => generated 409 bytes in 285 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 10:48:29.271008 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf374fdd-633f-4340-9f15-1b0f688c24ef admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:29.273692 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-bf374fdd-633f-4340-9f15-1b0f688c24ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:29.279814 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 942/1855] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 10:48:29 2022] PUT /identity/v3/projects/4d21e17346b343419d55a98ae4b9728f/users/ebf4e0c178664cb6913ed3698ef5c7ce/roles/21156560eb6b436bb567a40b4976f3f4 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:48:29.280862 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4beebb-5692-4eea-8880-814242dee0a7 admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:29.351671 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf374fdd-633f-4340-9f15-1b0f688c24ef admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:29.376609 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4beebb-5692-4eea-8880-814242dee0a7 admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:29.387029 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5279117f-dfb9-41d2-b911-7561aa628d86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:29.388155 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5279117f-dfb9-41d2-b911-7561aa628d86 admin admin] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:29.388584 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5279117f-dfb9-41d2-b911-7561aa628d86 admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:29.388854 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5279117f-dfb9-41d2-b911-7561aa628d86 admin admin] PATH_INFO: `/v3/users` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:29.390227 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5279117f-dfb9-41d2-b911-7561aa628d86 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:29.393058 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5279117f-dfb9-41d2-b911-7561aa628d86 admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:29.407363 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=86,keystone:DELETE=9,keystone:INSERT=5 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:48:29.408521 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f39f002-ea2d-4588-8c2c-f044fd9bc3dd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7922d0a1a7c4e808d0b2a80dca0da40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:29.409286 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f39f002-ea2d-4588-8c2c-f044fd9bc3dd service neutron] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:29.409618 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f39f002-ea2d-4588-8c2c-f044fd9bc3dd service neutron] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:29.409908 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f39f002-ea2d-4588-8c2c-f044fd9bc3dd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:29.423959 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 943/1856] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Jul 27 10:48:29 2022] POST /identity/v3/users => generated 311 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 10:48:29.425415 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7b23e1-5800-4d89-8ce4-ac0bdbbf8e93 None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:29.426054 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7b23e1-5800-4d89-8ce4-ac0bdbbf8e93 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:29.426443 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7b23e1-5800-4d89-8ce4-ac0bdbbf8e93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:29.436490 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f39f002-ea2d-4588-8c2c-f044fd9bc3dd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:29.438874 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f39f002-ea2d-4588-8c2c-f044fd9bc3dd service neutron] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:29.450911 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 914/1857] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:48:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 10:48:29.452443 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f39f002-ea2d-4588-8c2c-f044fd9bc3dd service neutron] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:29.460594 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-fd7b23e1-5800-4d89-8ce4-ac0bdbbf8e93 None None] MFA Rules not processed for user `ebf4e0c178664cb6913ed3698ef5c7ce`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:48:29.461948 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-fd7b23e1-5800-4d89-8ce4-ac0bdbbf8e93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:29.462152 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f39f002-ea2d-4588-8c2c-f044fd9bc3dd service neutron] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:29.468273 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e6bc63-dd47-4027-9139-aaf3ca6f4484 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:29.468538 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e6bc63-dd47-4027-9139-aaf3ca6f4484 admin admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:29.468830 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e6bc63-dd47-4027-9139-aaf3ca6f4484 admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:29.469115 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e6bc63-dd47-4027-9139-aaf3ca6f4484 admin admin] PATH_INFO: `/v3/roles` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:29.469815 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e6bc63-dd47-4027-9139-aaf3ca6f4484 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:29.471643 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e6bc63-dd47-4027-9139-aaf3ca6f4484 admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:29.482257 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 915/1858] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 10:48:29 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 10:48:29.486431 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e6bc63-dd47-4027-9139-aaf3ca6f4484 admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:29.498371 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e6bc63-dd47-4027-9139-aaf3ca6f4484 admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:29.506912 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d383c8-84c2-4dd2-ab2a-1ea9d2c0a7db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:29.507705 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d383c8-84c2-4dd2-ab2a-1ea9d2c0a7db admin admin] REQUEST_METHOD: `PUT` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:29.507905 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d383c8-84c2-4dd2-ab2a-1ea9d2c0a7db admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:29.508192 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d383c8-84c2-4dd2-ab2a-1ea9d2c0a7db admin admin] PATH_INFO: `/v3/projects/13d6ee0c8c9740c3a581b97b8f370525/users/838adbfb1b7b4b35bfd49dde18d5d9e4/roles/21156560eb6b436bb567a40b4976f3f4` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:29.545706 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d383c8-84c2-4dd2-ab2a-1ea9d2c0a7db admin admin] RBAC: Authorizing `identity:create_grant(project_id=13d6ee0c8c9740c3a581b97b8f370525, user_id=838adbfb1b7b4b35bfd49dde18d5d9e4, role_id=21156560eb6b436bb567a40b4976f3f4)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:29.549302 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d383c8-84c2-4dd2-ab2a-1ea9d2c0a7db admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:29.571308 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 944/1859] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:48:29 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:48:29.581926 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7b23e1-5800-4d89-8ce4-ac0bdbbf8e93 None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:29.582661 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 916/1860] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 10:48:29 2022] PUT /identity/v3/projects/13d6ee0c8c9740c3a581b97b8f370525/users/838adbfb1b7b4b35bfd49dde18d5d9e4/roles/21156560eb6b436bb567a40b4976f3f4 => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 10:48:29.588905 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d383c8-84c2-4dd2-ab2a-1ea9d2c0a7db admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:29.595068 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-e3d383c8-84c2-4dd2-ab2a-1ea9d2c0a7db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:29.602010 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7b23e1-5800-4d89-8ce4-ac0bdbbf8e93 None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:29.673002 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a47659-9a44-47e7-84ae-842b5f663830 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:29.674447 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a47659-9a44-47e7-84ae-842b5f663830 admin admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:29.674800 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a47659-9a44-47e7-84ae-842b5f663830 admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:29.675300 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a47659-9a44-47e7-84ae-842b5f663830 admin admin] PATH_INFO: `/v3/roles` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:29.676105 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a47659-9a44-47e7-84ae-842b5f663830 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:29.678096 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a47659-9a44-47e7-84ae-842b5f663830 admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:29.681090 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d383c8-84c2-4dd2-ab2a-1ea9d2c0a7db admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:29.684776 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f11722-77aa-4ae2-9b58-7ec20c103f46 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '864d35f268cf45148756109759bb2743', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:29.685808 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f11722-77aa-4ae2-9b58-7ec20c103f46 service cinder] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:29.686111 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f11722-77aa-4ae2-9b58-7ec20c103f46 service cinder] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:29.686399 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f11722-77aa-4ae2-9b58-7ec20c103f46 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:29.692517 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 945/1861] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 10:48:29 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 10:48:29.693605 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a47659-9a44-47e7-84ae-842b5f663830 admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:29.708506 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a47659-9a44-47e7-84ae-842b5f663830 admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:29.713554 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e76a2a-e686-4840-ac22-4316bef58dc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:29.714243 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e76a2a-e686-4840-ac22-4316bef58dc3 admin admin] REQUEST_METHOD: `PUT` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:29.714530 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e76a2a-e686-4840-ac22-4316bef58dc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:29.714846 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e76a2a-e686-4840-ac22-4316bef58dc3 admin admin] PATH_INFO: `/v3/projects/13d6ee0c8c9740c3a581b97b8f370525/users/838adbfb1b7b4b35bfd49dde18d5d9e4/roles/21156560eb6b436bb567a40b4976f3f4` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:29.717399 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e76a2a-e686-4840-ac22-4316bef58dc3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=13d6ee0c8c9740c3a581b97b8f370525, user_id=838adbfb1b7b4b35bfd49dde18d5d9e4, role_id=21156560eb6b436bb567a40b4976f3f4)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:29.720118 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e76a2a-e686-4840-ac22-4316bef58dc3 admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:29.725109 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:48:29.735050 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 946/1862] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 10:48:29 2022] PUT /identity/v3/projects/13d6ee0c8c9740c3a581b97b8f370525/users/838adbfb1b7b4b35bfd49dde18d5d9e4/roles/21156560eb6b436bb567a40b4976f3f4 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:48:29.746228 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e120a2a7-9e9f-4fc7-9a8e-1ce3d68a03ea None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:29.746664 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e120a2a7-9e9f-4fc7-9a8e-1ce3d68a03ea None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:29.747051 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e120a2a7-9e9f-4fc7-9a8e-1ce3d68a03ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:29.775652 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f11722-77aa-4ae2-9b58-7ec20c103f46 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:29.778403 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f11722-77aa-4ae2-9b58-7ec20c103f46 service cinder] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:29.780590 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-e120a2a7-9e9f-4fc7-9a8e-1ce3d68a03ea None None] MFA Rules not processed for user `838adbfb1b7b4b35bfd49dde18d5d9e4`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:48:29.782486 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-e120a2a7-9e9f-4fc7-9a8e-1ce3d68a03ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:29.792092 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 917/1863] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:48:29 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 10:48:29.855456 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f11722-77aa-4ae2-9b58-7ec20c103f46 service cinder] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:29.858394 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-d2f11722-77aa-4ae2-9b58-7ec20c103f46 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:29.860580 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:48:29.879884 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f11722-77aa-4ae2-9b58-7ec20c103f46 service cinder] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:29.900515 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 947/1864] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:48:29 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 10:48:29.930460 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e120a2a7-9e9f-4fc7-9a8e-1ce3d68a03ea None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:29.975287 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe4b6b1-2062-43c9-9bad-8d6b9fee0136 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '864d35f268cf45148756109759bb2743', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:29.975761 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe4b6b1-2062-43c9-9bad-8d6b9fee0136 service cinder] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:29.976191 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe4b6b1-2062-43c9-9bad-8d6b9fee0136 service cinder] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:29.976576 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe4b6b1-2062-43c9-9bad-8d6b9fee0136 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:29.978729 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-abe4b6b1-2062-43c9-9bad-8d6b9fee0136 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:30.020141 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e120a2a7-9e9f-4fc7-9a8e-1ce3d68a03ea None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:30.024004 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1641b529-4627-4b4f-824e-30f03e867cce service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '864d35f268cf45148756109759bb2743', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:30.024630 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1641b529-4627-4b4f-824e-30f03e867cce service cinder] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:30.024911 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1641b529-4627-4b4f-824e-30f03e867cce service cinder] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:30.025183 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1641b529-4627-4b4f-824e-30f03e867cce service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:30.035234 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1641b529-4627-4b4f-824e-30f03e867cce service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:30.036677 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1641b529-4627-4b4f-824e-30f03e867cce service cinder] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:30.047165 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 948/1865] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:48:29 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 10:48:30.051481 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe4b6b1-2062-43c9-9bad-8d6b9fee0136 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:30.053265 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe4b6b1-2062-43c9-9bad-8d6b9fee0136 service cinder] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:30.062915 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 918/1866] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:48:29 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 10:48:30.479269 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1641b529-4627-4b4f-824e-30f03e867cce service cinder] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:30.480867 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-1641b529-4627-4b4f-824e-30f03e867cce service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:30.635740 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe4b6b1-2062-43c9-9bad-8d6b9fee0136 service cinder] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:30.675691 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe4b6b1-2062-43c9-9bad-8d6b9fee0136 service cinder] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:30.694662 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1641b529-4627-4b4f-824e-30f03e867cce service cinder] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:30.695301 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0ee5dd-b35b-4d95-9c23-c3fd2e510d69 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '864d35f268cf45148756109759bb2743', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:30.695301 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0ee5dd-b35b-4d95-9c23-c3fd2e510d69 service cinder] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:30.695301 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0ee5dd-b35b-4d95-9c23-c3fd2e510d69 service cinder] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:30.695301 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0ee5dd-b35b-4d95-9c23-c3fd2e510d69 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:30.704040 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-6e0ee5dd-b35b-4d95-9c23-c3fd2e510d69 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:30.706644 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a854792d-d242-4744-9fc6-2a7c04f60ffd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:30.707319 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a854792d-d242-4744-9fc6-2a7c04f60ffd admin admin] REQUEST_METHOD: `DELETE` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:30.707601 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a854792d-d242-4744-9fc6-2a7c04f60ffd admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:30.707879 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a854792d-d242-4744-9fc6-2a7c04f60ffd admin admin] PATH_INFO: `/v3/projects/13f7a28998894dbbb487b4d71426e8b3` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:30.720165 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a854792d-d242-4744-9fc6-2a7c04f60ffd admin admin] RBAC: Authorizing `identity:delete_project(project_id=13f7a28998894dbbb487b4d71426e8b3)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:30.729613 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a854792d-d242-4744-9fc6-2a7c04f60ffd admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:30.862814 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-a854792d-d242-4744-9fc6-2a7c04f60ffd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '13f7a28998894dbbb487b4d71426e8b3', 'request_id': 'req-a854792d-d242-4744-9fc6-2a7c04f60ffd'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:30.879765 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-a854792d-d242-4744-9fc6-2a7c04f60ffd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '13f7a28998894dbbb487b4d71426e8b3', 'request_id': 'req-a854792d-d242-4744-9fc6-2a7c04f60ffd'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:30.906297 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-a854792d-d242-4744-9fc6-2a7c04f60ffd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '13f7a28998894dbbb487b4d71426e8b3', 'request_id': 'req-a854792d-d242-4744-9fc6-2a7c04f60ffd'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:30.914491 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0ee5dd-b35b-4d95-9c23-c3fd2e510d69 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:30.943980 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0ee5dd-b35b-4d95-9c23-c3fd2e510d69 service cinder] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:30.957737 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-a854792d-d242-4744-9fc6-2a7c04f60ffd admin admin] The token cache is being invalidate because project 13f7a28998894dbbb487b4d71426e8b3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86827) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 10:48:30.958073 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-a854792d-d242-4744-9fc6-2a7c04f60ffd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:30.958800 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-a854792d-d242-4744-9fc6-2a7c04f60ffd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:30.964578 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 949/1867] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Jul 27 10:48:30 2022] DELETE /identity/v3/projects/13f7a28998894dbbb487b4d71426e8b3 => generated 0 bytes in 486 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:48:30.973303 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a854792d-d242-4744-9fc6-2a7c04f60ffd admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:30.974657 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-a854792d-d242-4744-9fc6-2a7c04f60ffd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:30.991947 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 919/1868] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:48:30 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 10:48:31.041655 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:48:31.104330 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a854792d-d242-4744-9fc6-2a7c04f60ffd admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:31.109338 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0957d133-aa96-403f-b862-08b33f4deeef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:31.110010 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0957d133-aa96-403f-b862-08b33f4deeef admin admin] REQUEST_METHOD: `DELETE` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:31.110295 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0957d133-aa96-403f-b862-08b33f4deeef admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:31.110575 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0957d133-aa96-403f-b862-08b33f4deeef admin admin] PATH_INFO: `/v3/users/bc07a6fadb4944e19e1b5f2b4380e289` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:31.112223 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0957d133-aa96-403f-b862-08b33f4deeef admin admin] RBAC: Authorizing `identity:delete_user(user_id=bc07a6fadb4944e19e1b5f2b4380e289)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:31.114356 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0957d133-aa96-403f-b862-08b33f4deeef admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:31.194114 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-0957d133-aa96-403f-b862-08b33f4deeef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bc07a6fadb4944e19e1b5f2b4380e289'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:31.204022 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-0957d133-aa96-403f-b862-08b33f4deeef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bc07a6fadb4944e19e1b5f2b4380e289'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:31.217212 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-0957d133-aa96-403f-b862-08b33f4deeef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bc07a6fadb4944e19e1b5f2b4380e289'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:31.217212 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-0957d133-aa96-403f-b862-08b33f4deeef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bc07a6fadb4944e19e1b5f2b4380e289'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:31.217859 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-0957d133-aa96-403f-b862-08b33f4deeef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bc07a6fadb4944e19e1b5f2b4380e289'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:31.272934 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 950/1869] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Jul 27 10:48:30 2022] DELETE /identity/v3/users/bc07a6fadb4944e19e1b5f2b4380e289 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:48:32.478016 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0ee5dd-b35b-4d95-9c23-c3fd2e510d69 service cinder] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:32.492072 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0ee5dd-b35b-4d95-9c23-c3fd2e510d69 service cinder] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:32.512539 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:48:32.561238 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2013e573-c277-4f88-91fc-cf427fcb58bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:32.561890 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2013e573-c277-4f88-91fc-cf427fcb58bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:32.562288 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2013e573-c277-4f88-91fc-cf427fcb58bc admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:32.562573 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2013e573-c277-4f88-91fc-cf427fcb58bc admin admin] PATH_INFO: `/v3/projects/eff6b0631db74f47816a6669bc1e0368` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:32.563769 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2013e573-c277-4f88-91fc-cf427fcb58bc admin admin] RBAC: Authorizing `identity:delete_project(project_id=eff6b0631db74f47816a6669bc1e0368)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:32.565663 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2013e573-c277-4f88-91fc-cf427fcb58bc admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:32.612050 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-2013e573-c277-4f88-91fc-cf427fcb58bc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eff6b0631db74f47816a6669bc1e0368', 'request_id': 'req-2013e573-c277-4f88-91fc-cf427fcb58bc'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:32.618917 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-2013e573-c277-4f88-91fc-cf427fcb58bc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eff6b0631db74f47816a6669bc1e0368', 'request_id': 'req-2013e573-c277-4f88-91fc-cf427fcb58bc'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:32.628889 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-2013e573-c277-4f88-91fc-cf427fcb58bc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eff6b0631db74f47816a6669bc1e0368', 'request_id': 'req-2013e573-c277-4f88-91fc-cf427fcb58bc'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:32.640633 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-2013e573-c277-4f88-91fc-cf427fcb58bc admin admin] The token cache is being invalidate because project eff6b0631db74f47816a6669bc1e0368 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86828) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 10:48:32.640932 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-2013e573-c277-4f88-91fc-cf427fcb58bc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:32.641604 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-2013e573-c277-4f88-91fc-cf427fcb58bc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:32.643189 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 920/1870] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Jul 27 10:48:32 2022] DELETE /identity/v3/projects/eff6b0631db74f47816a6669bc1e0368 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:48:32.649371 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dbbbbc-5083-4caa-8c60-98ed3c99344c None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:32.649848 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dbbbbc-5083-4caa-8c60-98ed3c99344c None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:32.649937 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dbbbbc-5083-4caa-8c60-98ed3c99344c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:32.672626 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-38dbbbbc-5083-4caa-8c60-98ed3c99344c None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:48:32.674072 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-38dbbbbc-5083-4caa-8c60-98ed3c99344c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:32.742586 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 951/1871] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:48:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:48:32.750425 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92afb41-9458-4534-9bdc-cb0ab381ae5f None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:32.750811 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92afb41-9458-4534-9bdc-cb0ab381ae5f None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:32.751101 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92afb41-9458-4534-9bdc-cb0ab381ae5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:32.783370 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-d92afb41-9458-4534-9bdc-cb0ab381ae5f None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:48:32.784846 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-d92afb41-9458-4534-9bdc-cb0ab381ae5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:32.799372 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 921/1872] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:48:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:48:32.803142 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38dbbbbc-5083-4caa-8c60-98ed3c99344c None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:32.817506 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38dbbbbc-5083-4caa-8c60-98ed3c99344c None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:32.833074 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cbc882-2b79-4f24-b9ed-5abcd39fdf8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:32.833885 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cbc882-2b79-4f24-b9ed-5abcd39fdf8f admin admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:32.834293 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cbc882-2b79-4f24-b9ed-5abcd39fdf8f admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:32.834627 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cbc882-2b79-4f24-b9ed-5abcd39fdf8f admin admin] PATH_INFO: `/v3/domains` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:32.835618 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81cbc882-2b79-4f24-b9ed-5abcd39fdf8f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86827) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:48:32.835618 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81cbc882-2b79-4f24-b9ed-5abcd39fdf8f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:32.841915 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81cbc882-2b79-4f24-b9ed-5abcd39fdf8f admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:32.867259 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92afb41-9458-4534-9bdc-cb0ab381ae5f None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:32.873653 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 952/1873] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Jul 27 10:48:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 10:48:32.878031 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92afb41-9458-4534-9bdc-cb0ab381ae5f None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:32.883895 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01b7515-00e5-4cd0-92fc-20a957e35dad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:32.884526 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01b7515-00e5-4cd0-92fc-20a957e35dad admin admin] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:32.884807 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01b7515-00e5-4cd0-92fc-20a957e35dad admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:32.885142 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01b7515-00e5-4cd0-92fc-20a957e35dad admin admin] PATH_INFO: `/v3/projects` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:32.885808 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01b7515-00e5-4cd0-92fc-20a957e35dad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:32.888446 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01b7515-00e5-4cd0-92fc-20a957e35dad admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:32.940140 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 922/1874] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Jul 27 10:48:32 2022] POST /identity/v3/projects => generated 397 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 10:48:32.944654 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cbc882-2b79-4f24-b9ed-5abcd39fdf8f admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:32.971054 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cbc882-2b79-4f24-b9ed-5abcd39fdf8f admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:33.054189 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:48:33.133190 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cd085d-de0c-445d-aeeb-d4b214d1594b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:33.134359 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cd085d-de0c-445d-aeeb-d4b214d1594b admin admin] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:33.134789 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cd085d-de0c-445d-aeeb-d4b214d1594b admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:33.135144 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cd085d-de0c-445d-aeeb-d4b214d1594b admin admin] PATH_INFO: `/v3/users` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:33.136409 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7cd085d-de0c-445d-aeeb-d4b214d1594b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:33.140279 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7cd085d-de0c-445d-aeeb-d4b214d1594b admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:33.178693 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 953/1875] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Jul 27 10:48:32 2022] POST /identity/v3/users => generated 299 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 10:48:33.178842 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01b7515-00e5-4cd0-92fc-20a957e35dad admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:33.191381 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01b7515-00e5-4cd0-92fc-20a957e35dad admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:33.198122 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafea57b-1910-4c62-9899-e3ab2b3576be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:33.199151 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafea57b-1910-4c62-9899-e3ab2b3576be admin admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:33.200009 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafea57b-1910-4c62-9899-e3ab2b3576be admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:33.200289 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafea57b-1910-4c62-9899-e3ab2b3576be admin admin] PATH_INFO: `/v3/roles` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:33.201011 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafea57b-1910-4c62-9899-e3ab2b3576be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:33.202724 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafea57b-1910-4c62-9899-e3ab2b3576be admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:33.215158 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 923/1876] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 10:48:33 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 10:48:33.219933 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cd085d-de0c-445d-aeeb-d4b214d1594b admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:33.233546 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cd085d-de0c-445d-aeeb-d4b214d1594b admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:33.239955 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4654dcf-db23-4e42-ab5e-6d9cec14d99d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:33.240784 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4654dcf-db23-4e42-ab5e-6d9cec14d99d admin admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:33.241146 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4654dcf-db23-4e42-ab5e-6d9cec14d99d admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:33.241586 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4654dcf-db23-4e42-ab5e-6d9cec14d99d admin admin] PATH_INFO: `/v3/roles` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:33.242369 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4654dcf-db23-4e42-ab5e-6d9cec14d99d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:33.244310 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4654dcf-db23-4e42-ab5e-6d9cec14d99d admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:33.263912 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 954/1877] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 10:48:33 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 10:48:33.269389 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafea57b-1910-4c62-9899-e3ab2b3576be admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:33.287999 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafea57b-1910-4c62-9899-e3ab2b3576be admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:33.294216 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334446e3-7d04-48c1-993d-5619766a19b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:33.295500 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334446e3-7d04-48c1-993d-5619766a19b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:33.295888 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334446e3-7d04-48c1-993d-5619766a19b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:33.296267 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334446e3-7d04-48c1-993d-5619766a19b1 admin admin] PATH_INFO: `/v3/projects/d571ca9b647f483a9663b6cf4872816a/users/5194030f71bc49ca94932bfe21f1db56/roles/4c9fc0b3baed441da2126be41cc6b04a` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:33.313497 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334446e3-7d04-48c1-993d-5619766a19b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d571ca9b647f483a9663b6cf4872816a, user_id=5194030f71bc49ca94932bfe21f1db56, role_id=4c9fc0b3baed441da2126be41cc6b04a)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:33.316596 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334446e3-7d04-48c1-993d-5619766a19b1 admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:33.332744 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 924/1878] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 10:48:33 2022] PUT /identity/v3/projects/d571ca9b647f483a9663b6cf4872816a/users/5194030f71bc49ca94932bfe21f1db56/roles/4c9fc0b3baed441da2126be41cc6b04a => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:48:33.343724 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a38412-f702-484e-9148-e7023265010c None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:33.344098 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a38412-f702-484e-9148-e7023265010c None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:33.344494 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a38412-f702-484e-9148-e7023265010c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:33.385237 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-a2a38412-f702-484e-9148-e7023265010c None None] MFA Rules not processed for user `5194030f71bc49ca94932bfe21f1db56`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:48:33.386647 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-a2a38412-f702-484e-9148-e7023265010c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:33.497286 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 955/1879] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:48:33 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:48:33.501963 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334446e3-7d04-48c1-993d-5619766a19b1 admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:33.582558 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:48:33.606988 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334446e3-7d04-48c1-993d-5619766a19b1 admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:33.613801 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46aec1bc-96de-442d-8d04-6ce11142f84c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:33.614687 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46aec1bc-96de-442d-8d04-6ce11142f84c admin admin] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:33.615043 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46aec1bc-96de-442d-8d04-6ce11142f84c admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:33.615421 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46aec1bc-96de-442d-8d04-6ce11142f84c admin admin] PATH_INFO: `/v3/projects` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:33.616487 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46aec1bc-96de-442d-8d04-6ce11142f84c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:33.619658 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46aec1bc-96de-442d-8d04-6ce11142f84c admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:33.666547 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 925/1880] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Jul 27 10:48:33 2022] POST /identity/v3/projects => generated 401 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 10:48:33.667700 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a38412-f702-484e-9148-e7023265010c None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:33.686083 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a38412-f702-484e-9148-e7023265010c None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:33.707286 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46aec1bc-96de-442d-8d04-6ce11142f84c admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:33.709834 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-46aec1bc-96de-442d-8d04-6ce11142f84c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:33.784901 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a68561-4ece-4455-905a-985fff9bb2ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:33.785526 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a68561-4ece-4455-905a-985fff9bb2ae admin admin] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:33.785807 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a68561-4ece-4455-905a-985fff9bb2ae admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:33.786081 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a68561-4ece-4455-905a-985fff9bb2ae admin admin] PATH_INFO: `/v3/users` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:33.786778 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a68561-4ece-4455-905a-985fff9bb2ae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:33.788704 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a68561-4ece-4455-905a-985fff9bb2ae admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:33.789901 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46aec1bc-96de-442d-8d04-6ce11142f84c admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:33.794836 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529f718a-82a4-4531-8718-354f9e31ae62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:33.795580 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529f718a-82a4-4531-8718-354f9e31ae62 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:33.796031 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529f718a-82a4-4531-8718-354f9e31ae62 admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:33.796379 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529f718a-82a4-4531-8718-354f9e31ae62 admin admin] PATH_INFO: `/v3/users/10087ffcfc514c3e8400ff5c91ce5ceb` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:33.797684 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529f718a-82a4-4531-8718-354f9e31ae62 admin admin] RBAC: Authorizing `identity:delete_user(user_id=10087ffcfc514c3e8400ff5c91ce5ceb)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:33.799866 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529f718a-82a4-4531-8718-354f9e31ae62 admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:33.813698 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 956/1881] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Jul 27 10:48:33 2022] POST /identity/v3/users => generated 307 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 10:48:33.817697 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a68561-4ece-4455-905a-985fff9bb2ae admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:33.833601 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a68561-4ece-4455-905a-985fff9bb2ae admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:33.841440 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77300c9f-c98a-411b-942b-49ea53a1c5c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:33.842077 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77300c9f-c98a-411b-942b-49ea53a1c5c6 admin admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:33.842362 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77300c9f-c98a-411b-942b-49ea53a1c5c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:33.842668 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77300c9f-c98a-411b-942b-49ea53a1c5c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:33.843374 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77300c9f-c98a-411b-942b-49ea53a1c5c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:33.844985 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77300c9f-c98a-411b-942b-49ea53a1c5c6 admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:33.855593 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 957/1882] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 10:48:33 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 10:48:33.856409 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77300c9f-c98a-411b-942b-49ea53a1c5c6 admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:33.858142 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-77300c9f-c98a-411b-942b-49ea53a1c5c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:33.860958 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-529f718a-82a4-4531-8718-354f9e31ae62 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '10087ffcfc514c3e8400ff5c91ce5ceb'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:33.868014 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-529f718a-82a4-4531-8718-354f9e31ae62 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '10087ffcfc514c3e8400ff5c91ce5ceb'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:33.870428 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77300c9f-c98a-411b-942b-49ea53a1c5c6 admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:33.875727 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecaa8f9-cdf5-4f2d-a291-edd734616c87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:33.875957 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-529f718a-82a4-4531-8718-354f9e31ae62 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '10087ffcfc514c3e8400ff5c91ce5ceb'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:33.876352 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecaa8f9-cdf5-4f2d-a291-edd734616c87 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:33.876594 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-529f718a-82a4-4531-8718-354f9e31ae62 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '10087ffcfc514c3e8400ff5c91ce5ceb'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:33.876973 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecaa8f9-cdf5-4f2d-a291-edd734616c87 admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:33.877155 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecaa8f9-cdf5-4f2d-a291-edd734616c87 admin admin] PATH_INFO: `/v3/users/3efff22ca50741bc9e4d2b1f426f3d7f` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:33.877599 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-529f718a-82a4-4531-8718-354f9e31ae62 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '10087ffcfc514c3e8400ff5c91ce5ceb'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:33.878317 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecaa8f9-cdf5-4f2d-a291-edd734616c87 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3efff22ca50741bc9e4d2b1f426f3d7f)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:33.880340 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecaa8f9-cdf5-4f2d-a291-edd734616c87 admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:33.891853 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:48:33.896969 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 926/1883] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Jul 27 10:48:33 2022] DELETE /identity/v3/users/10087ffcfc514c3e8400ff5c91ce5ceb => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 10:48:33.897516 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529f718a-82a4-4531-8718-354f9e31ae62 admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:33.898862 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-529f718a-82a4-4531-8718-354f9e31ae62 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:33.928529 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-4ecaa8f9-cdf5-4f2d-a291-edd734616c87 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3efff22ca50741bc9e4d2b1f426f3d7f'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:33.936854 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-4ecaa8f9-cdf5-4f2d-a291-edd734616c87 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3efff22ca50741bc9e4d2b1f426f3d7f'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:33.945418 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-4ecaa8f9-cdf5-4f2d-a291-edd734616c87 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3efff22ca50741bc9e4d2b1f426f3d7f'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:33.946468 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-4ecaa8f9-cdf5-4f2d-a291-edd734616c87 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3efff22ca50741bc9e4d2b1f426f3d7f'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:33.947246 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-4ecaa8f9-cdf5-4f2d-a291-edd734616c87 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3efff22ca50741bc9e4d2b1f426f3d7f'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:33.954353 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:48:33.967267 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 958/1884] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Jul 27 10:48:33 2022] DELETE /identity/v3/users/3efff22ca50741bc9e4d2b1f426f3d7f => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:48:33.968765 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529f718a-82a4-4531-8718-354f9e31ae62 admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:33.971459 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b804da2-f91c-4ea7-b2be-14cd29aa318b None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:33.971759 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b804da2-f91c-4ea7-b2be-14cd29aa318b None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:33.972038 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b804da2-f91c-4ea7-b2be-14cd29aa318b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:33.990967 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-6b804da2-f91c-4ea7-b2be-14cd29aa318b None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:48:33.992313 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-6b804da2-f91c-4ea7-b2be-14cd29aa318b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:34.035024 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09f00cc-76d1-4b75-a519-ee194b9835b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:34.035720 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09f00cc-76d1-4b75-a519-ee194b9835b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:34.035999 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09f00cc-76d1-4b75-a519-ee194b9835b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:34.036632 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09f00cc-76d1-4b75-a519-ee194b9835b7 admin admin] PATH_INFO: `/v3/projects/dd36bbdd9cde482b9c79b928ddc8b0e7/users/d51214bfb27e4f0696f0c66411f35fe9/roles/21156560eb6b436bb567a40b4976f3f4` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:34.055602 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09f00cc-76d1-4b75-a519-ee194b9835b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd36bbdd9cde482b9c79b928ddc8b0e7, user_id=d51214bfb27e4f0696f0c66411f35fe9, role_id=21156560eb6b436bb567a40b4976f3f4)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:34.059833 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09f00cc-76d1-4b75-a519-ee194b9835b7 admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:34.064026 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 959/1885] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:48:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:48:34.065865 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbf588d-73f4-47cb-98fb-2a5b51f8ac91 None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:34.066475 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbf588d-73f4-47cb-98fb-2a5b51f8ac91 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:34.066997 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbf588d-73f4-47cb-98fb-2a5b51f8ac91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:34.082030 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 927/1886] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 10:48:33 2022] PUT /identity/v3/projects/dd36bbdd9cde482b9c79b928ddc8b0e7/users/d51214bfb27e4f0696f0c66411f35fe9/roles/21156560eb6b436bb567a40b4976f3f4 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:48:34.086213 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09f00cc-76d1-4b75-a519-ee194b9835b7 admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:34.095697 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-b09f00cc-76d1-4b75-a519-ee194b9835b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:34.097767 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-ffbf588d-73f4-47cb-98fb-2a5b51f8ac91 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:48:34.099108 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-ffbf588d-73f4-47cb-98fb-2a5b51f8ac91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:34.208933 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09f00cc-76d1-4b75-a519-ee194b9835b7 admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:34.214492 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1209ad6d-3705-4e63-8ab0-9880ad21c272 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7922d0a1a7c4e808d0b2a80dca0da40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:34.215523 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 960/1887] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:48:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:48:34.218851 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbf588d-73f4-47cb-98fb-2a5b51f8ac91 None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:34.219230 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1209ad6d-3705-4e63-8ab0-9880ad21c272 service neutron] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:34.219829 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-ffbf588d-73f4-47cb-98fb-2a5b51f8ac91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:34.220297 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1209ad6d-3705-4e63-8ab0-9880ad21c272 service neutron] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:34.220572 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1209ad6d-3705-4e63-8ab0-9880ad21c272 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:34.231662 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1209ad6d-3705-4e63-8ab0-9880ad21c272 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:34.234773 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1209ad6d-3705-4e63-8ab0-9880ad21c272 service neutron] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:34.234925 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbf588d-73f4-47cb-98fb-2a5b51f8ac91 None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:34.241105 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb4dabd-9f18-4f32-b4fc-62a405fa545d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:34.241951 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb4dabd-9f18-4f32-b4fc-62a405fa545d admin admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:34.242840 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb4dabd-9f18-4f32-b4fc-62a405fa545d admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:34.243130 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb4dabd-9f18-4f32-b4fc-62a405fa545d admin admin] PATH_INFO: `/v3/roles` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:34.243849 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb4dabd-9f18-4f32-b4fc-62a405fa545d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:34.245905 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb4dabd-9f18-4f32-b4fc-62a405fa545d admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:34.249212 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 928/1888] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:48:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 10:48:34.250133 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1209ad6d-3705-4e63-8ab0-9880ad21c272 service neutron] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:34.260442 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 961/1889] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 10:48:34 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 10:48:34.262974 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1209ad6d-3705-4e63-8ab0-9880ad21c272 service neutron] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:34.267640 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d44e06c-eb07-411e-a339-67cf039d8b7d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7922d0a1a7c4e808d0b2a80dca0da40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:34.268416 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d44e06c-eb07-411e-a339-67cf039d8b7d service neutron] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:34.268702 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d44e06c-eb07-411e-a339-67cf039d8b7d service neutron] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:34.269051 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d44e06c-eb07-411e-a339-67cf039d8b7d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:34.271590 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb4dabd-9f18-4f32-b4fc-62a405fa545d admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:34.279910 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d44e06c-eb07-411e-a339-67cf039d8b7d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:34.281349 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d44e06c-eb07-411e-a339-67cf039d8b7d service neutron] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:34.284511 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb4dabd-9f18-4f32-b4fc-62a405fa545d admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:34.290637 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9b52cb-1f7b-4ea8-a876-aff0d6491c19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:34.291338 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9b52cb-1f7b-4ea8-a876-aff0d6491c19 admin admin] REQUEST_METHOD: `PUT` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:34.291621 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9b52cb-1f7b-4ea8-a876-aff0d6491c19 admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:34.291897 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9b52cb-1f7b-4ea8-a876-aff0d6491c19 admin admin] PATH_INFO: `/v3/projects/dd36bbdd9cde482b9c79b928ddc8b0e7/users/d51214bfb27e4f0696f0c66411f35fe9/roles/21156560eb6b436bb567a40b4976f3f4` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:34.293709 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 929/1890] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:48:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 10:48:34.294486 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9b52cb-1f7b-4ea8-a876-aff0d6491c19 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd36bbdd9cde482b9c79b928ddc8b0e7, user_id=d51214bfb27e4f0696f0c66411f35fe9, role_id=21156560eb6b436bb567a40b4976f3f4)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:34.298324 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9b52cb-1f7b-4ea8-a876-aff0d6491c19 admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:34.318659 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 962/1891] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 10:48:34 2022] PUT /identity/v3/projects/dd36bbdd9cde482b9c79b928ddc8b0e7/users/d51214bfb27e4f0696f0c66411f35fe9/roles/21156560eb6b436bb567a40b4976f3f4 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:48:34.323672 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347a1c35-c943-46ff-aa76-97ec22d5673c None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:34.324109 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347a1c35-c943-46ff-aa76-97ec22d5673c None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:34.324500 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347a1c35-c943-46ff-aa76-97ec22d5673c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:34.366229 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-347a1c35-c943-46ff-aa76-97ec22d5673c None None] MFA Rules not processed for user `d51214bfb27e4f0696f0c66411f35fe9`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:48:34.369329 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-347a1c35-c943-46ff-aa76-97ec22d5673c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:34.458293 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:48:34.481409 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 930/1892] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:48:34 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:48:34.482819 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9b52cb-1f7b-4ea8-a876-aff0d6491c19 admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:34.487201 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-1f9b52cb-1f7b-4ea8-a876-aff0d6491c19 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:34.502888 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:DELETE=10,keystone:SELECT=88,keystone:INSERT=2 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:48:34.582468 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9b52cb-1f7b-4ea8-a876-aff0d6491c19 admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:34.588589 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0856f735-329d-4097-85b9-d36de9786200 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '864d35f268cf45148756109759bb2743', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:34.589004 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0856f735-329d-4097-85b9-d36de9786200 service cinder] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:34.589353 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0856f735-329d-4097-85b9-d36de9786200 service cinder] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:34.590574 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0856f735-329d-4097-85b9-d36de9786200 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:34.602165 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0856f735-329d-4097-85b9-d36de9786200 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:34.603966 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0856f735-329d-4097-85b9-d36de9786200 service cinder] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:34.616447 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 963/1893] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:48:34 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 10:48:34.757686 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347a1c35-c943-46ff-aa76-97ec22d5673c None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:34.759337 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-347a1c35-c943-46ff-aa76-97ec22d5673c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:34.789232 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347a1c35-c943-46ff-aa76-97ec22d5673c None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:34.800446 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d520edf-cf23-42d3-b2c6-25ea9ef3286a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '864d35f268cf45148756109759bb2743', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:34.801367 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d520edf-cf23-42d3-b2c6-25ea9ef3286a service cinder] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:34.801867 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d520edf-cf23-42d3-b2c6-25ea9ef3286a service cinder] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:34.802222 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d520edf-cf23-42d3-b2c6-25ea9ef3286a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:34.807405 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-8d520edf-cf23-42d3-b2c6-25ea9ef3286a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:34.821909 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0856f735-329d-4097-85b9-d36de9786200 service cinder] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:34.823502 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-0856f735-329d-4097-85b9-d36de9786200 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:34.878574 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d520edf-cf23-42d3-b2c6-25ea9ef3286a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:34.880526 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d520edf-cf23-42d3-b2c6-25ea9ef3286a service cinder] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:34.903511 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 931/1894] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:48:34 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 10:48:34.953918 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0856f735-329d-4097-85b9-d36de9786200 service cinder] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:34.959794 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1207f54-1365-47de-98ff-e5ab99f2078f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:34.960545 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1207f54-1365-47de-98ff-e5ab99f2078f admin admin] REQUEST_METHOD: `DELETE` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:34.960891 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1207f54-1365-47de-98ff-e5ab99f2078f admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:34.961182 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1207f54-1365-47de-98ff-e5ab99f2078f admin admin] PATH_INFO: `/v3/projects/ca74ae0b9f354340b1e8f54eca2450c4` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:34.962392 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1207f54-1365-47de-98ff-e5ab99f2078f admin admin] RBAC: Authorizing `identity:delete_project(project_id=ca74ae0b9f354340b1e8f54eca2450c4)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:34.964395 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1207f54-1365-47de-98ff-e5ab99f2078f admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:34.983691 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d520edf-cf23-42d3-b2c6-25ea9ef3286a service cinder] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:34.987390 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-8d520edf-cf23-42d3-b2c6-25ea9ef3286a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:35.010803 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d520edf-cf23-42d3-b2c6-25ea9ef3286a service cinder] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:35.024438 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-d1207f54-1365-47de-98ff-e5ab99f2078f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ca74ae0b9f354340b1e8f54eca2450c4', 'request_id': 'req-d1207f54-1365-47de-98ff-e5ab99f2078f'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:35.032788 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-d1207f54-1365-47de-98ff-e5ab99f2078f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ca74ae0b9f354340b1e8f54eca2450c4', 'request_id': 'req-d1207f54-1365-47de-98ff-e5ab99f2078f'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:35.057060 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-d1207f54-1365-47de-98ff-e5ab99f2078f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ca74ae0b9f354340b1e8f54eca2450c4', 'request_id': 'req-d1207f54-1365-47de-98ff-e5ab99f2078f'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:35.077067 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-d1207f54-1365-47de-98ff-e5ab99f2078f admin admin] The token cache is being invalidate because project ca74ae0b9f354340b1e8f54eca2450c4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86827) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 10:48:35.077373 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-d1207f54-1365-47de-98ff-e5ab99f2078f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:35.078033 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-d1207f54-1365-47de-98ff-e5ab99f2078f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:35.080517 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 964/1895] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Jul 27 10:48:34 2022] DELETE /identity/v3/projects/ca74ae0b9f354340b1e8f54eca2450c4 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:48:35.084489 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1207f54-1365-47de-98ff-e5ab99f2078f admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:35.086174 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-d1207f54-1365-47de-98ff-e5ab99f2078f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:35.134975 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb217d3-9736-49ca-bde1-872b33de4a99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:35.135717 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb217d3-9736-49ca-bde1-872b33de4a99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:35.136065 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb217d3-9736-49ca-bde1-872b33de4a99 admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:35.136408 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb217d3-9736-49ca-bde1-872b33de4a99 admin admin] PATH_INFO: `/v3/projects/0e1feac13d044638956a0d30726adddd` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:35.138477 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb217d3-9736-49ca-bde1-872b33de4a99 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0e1feac13d044638956a0d30726adddd)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:35.140722 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb217d3-9736-49ca-bde1-872b33de4a99 admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:35.143183 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:48:35.178899 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1207f54-1365-47de-98ff-e5ab99f2078f admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:35.185213 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833e1eb8-4bcc-4425-b75b-083377efa04c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:35.185948 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833e1eb8-4bcc-4425-b75b-083377efa04c admin admin] REQUEST_METHOD: `DELETE` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:35.186306 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833e1eb8-4bcc-4425-b75b-083377efa04c admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:35.188105 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833e1eb8-4bcc-4425-b75b-083377efa04c admin admin] PATH_INFO: `/v3/users/cf861046cc254a48b648f35b5d9e83e6` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:35.188105 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833e1eb8-4bcc-4425-b75b-083377efa04c admin admin] RBAC: Authorizing `identity:delete_user(user_id=cf861046cc254a48b648f35b5d9e83e6)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:35.191550 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833e1eb8-4bcc-4425-b75b-083377efa04c admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:35.231868 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-ceb217d3-9736-49ca-bde1-872b33de4a99 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0e1feac13d044638956a0d30726adddd', 'request_id': 'req-ceb217d3-9736-49ca-bde1-872b33de4a99'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:35.241812 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-ceb217d3-9736-49ca-bde1-872b33de4a99 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0e1feac13d044638956a0d30726adddd', 'request_id': 'req-ceb217d3-9736-49ca-bde1-872b33de4a99'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:35.253709 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-ceb217d3-9736-49ca-bde1-872b33de4a99 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0e1feac13d044638956a0d30726adddd', 'request_id': 'req-ceb217d3-9736-49ca-bde1-872b33de4a99'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:35.268236 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-ceb217d3-9736-49ca-bde1-872b33de4a99 admin admin] The token cache is being invalidate because project 0e1feac13d044638956a0d30726adddd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86828) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 10:48:35.268716 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-ceb217d3-9736-49ca-bde1-872b33de4a99 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:35.269408 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-ceb217d3-9736-49ca-bde1-872b33de4a99 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:35.272225 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 932/1896] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Jul 27 10:48:34 2022] DELETE /identity/v3/projects/0e1feac13d044638956a0d30726adddd => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 10:48:35.273059 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-833e1eb8-4bcc-4425-b75b-083377efa04c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cf861046cc254a48b648f35b5d9e83e6'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:35.276673 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb217d3-9736-49ca-bde1-872b33de4a99 admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:35.279795 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-ceb217d3-9736-49ca-bde1-872b33de4a99 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:35.281135 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-833e1eb8-4bcc-4425-b75b-083377efa04c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cf861046cc254a48b648f35b5d9e83e6'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:35.292646 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-833e1eb8-4bcc-4425-b75b-083377efa04c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cf861046cc254a48b648f35b5d9e83e6'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:35.294462 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-833e1eb8-4bcc-4425-b75b-083377efa04c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cf861046cc254a48b648f35b5d9e83e6'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:35.297633 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-833e1eb8-4bcc-4425-b75b-083377efa04c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cf861046cc254a48b648f35b5d9e83e6'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:35.339730 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:48:35.343540 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 965/1897] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Jul 27 10:48:35 2022] DELETE /identity/v3/users/cf861046cc254a48b648f35b5d9e83e6 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:48:35.388831 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb217d3-9736-49ca-bde1-872b33de4a99 admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:35.457372 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89048886-c991-4fee-84cb-5a9ead861676 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:35.458031 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89048886-c991-4fee-84cb-5a9ead861676 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:35.458322 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89048886-c991-4fee-84cb-5a9ead861676 admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:35.458633 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89048886-c991-4fee-84cb-5a9ead861676 admin admin] PATH_INFO: `/v3/users/adedde348d9540b6997d1108513c702d` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:35.459827 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89048886-c991-4fee-84cb-5a9ead861676 admin admin] RBAC: Authorizing `identity:delete_user(user_id=adedde348d9540b6997d1108513c702d)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:35.461789 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89048886-c991-4fee-84cb-5a9ead861676 admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:35.518874 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-89048886-c991-4fee-84cb-5a9ead861676 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'adedde348d9540b6997d1108513c702d'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:35.525605 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-89048886-c991-4fee-84cb-5a9ead861676 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'adedde348d9540b6997d1108513c702d'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:35.533765 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-89048886-c991-4fee-84cb-5a9ead861676 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'adedde348d9540b6997d1108513c702d'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:35.534548 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-89048886-c991-4fee-84cb-5a9ead861676 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'adedde348d9540b6997d1108513c702d'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:35.535286 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-89048886-c991-4fee-84cb-5a9ead861676 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'adedde348d9540b6997d1108513c702d'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:35.554416 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 933/1898] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Jul 27 10:48:35 2022] DELETE /identity/v3/users/adedde348d9540b6997d1108513c702d => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 10:48:35.617926 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833e1eb8-4bcc-4425-b75b-083377efa04c admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:35.619547 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-833e1eb8-4bcc-4425-b75b-083377efa04c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:35.634792 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833e1eb8-4bcc-4425-b75b-083377efa04c admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:35.711849 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13e580b-525d-4ad3-b938-f95a9d09d4c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:35.712509 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13e580b-525d-4ad3-b938-f95a9d09d4c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:35.712787 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13e580b-525d-4ad3-b938-f95a9d09d4c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:35.713061 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13e580b-525d-4ad3-b938-f95a9d09d4c5 admin admin] PATH_INFO: `/v3/projects/2499a2c444f94b91a8d2ac27daccf392` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:35.714278 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13e580b-525d-4ad3-b938-f95a9d09d4c5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2499a2c444f94b91a8d2ac27daccf392)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:35.716253 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13e580b-525d-4ad3-b938-f95a9d09d4c5 admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:35.772681 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-b13e580b-525d-4ad3-b938-f95a9d09d4c5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2499a2c444f94b91a8d2ac27daccf392', 'request_id': 'req-b13e580b-525d-4ad3-b938-f95a9d09d4c5'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:35.779830 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-b13e580b-525d-4ad3-b938-f95a9d09d4c5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2499a2c444f94b91a8d2ac27daccf392', 'request_id': 'req-b13e580b-525d-4ad3-b938-f95a9d09d4c5'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:35.784490 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:48:35.794677 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-b13e580b-525d-4ad3-b938-f95a9d09d4c5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2499a2c444f94b91a8d2ac27daccf392', 'request_id': 'req-b13e580b-525d-4ad3-b938-f95a9d09d4c5'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:35.813562 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-b13e580b-525d-4ad3-b938-f95a9d09d4c5 admin admin] The token cache is being invalidate because project 2499a2c444f94b91a8d2ac27daccf392 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86827) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 10:48:35.814074 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-b13e580b-525d-4ad3-b938-f95a9d09d4c5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:35.814969 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-b13e580b-525d-4ad3-b938-f95a9d09d4c5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:35.817432 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 966/1899] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Jul 27 10:48:35 2022] DELETE /identity/v3/projects/2499a2c444f94b91a8d2ac27daccf392 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:48:35.823516 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89048886-c991-4fee-84cb-5a9ead861676 admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:35.825438 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-89048886-c991-4fee-84cb-5a9ead861676 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:35.841962 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89048886-c991-4fee-84cb-5a9ead861676 admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:35.894225 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:48:35.908288 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ba8488-0cbc-4479-b5b5-3c2dc2bb14bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:35.910474 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ba8488-0cbc-4479-b5b5-3c2dc2bb14bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:35.910474 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ba8488-0cbc-4479-b5b5-3c2dc2bb14bb admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:35.910474 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ba8488-0cbc-4479-b5b5-3c2dc2bb14bb admin admin] PATH_INFO: `/v3/users/ebf4e0c178664cb6913ed3698ef5c7ce` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:35.911399 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ba8488-0cbc-4479-b5b5-3c2dc2bb14bb admin admin] RBAC: Authorizing `identity:delete_user(user_id=ebf4e0c178664cb6913ed3698ef5c7ce)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:35.913497 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ba8488-0cbc-4479-b5b5-3c2dc2bb14bb admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:35.988470 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-93ba8488-0cbc-4479-b5b5-3c2dc2bb14bb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ebf4e0c178664cb6913ed3698ef5c7ce'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:35.998618 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-93ba8488-0cbc-4479-b5b5-3c2dc2bb14bb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ebf4e0c178664cb6913ed3698ef5c7ce'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:36.012477 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-93ba8488-0cbc-4479-b5b5-3c2dc2bb14bb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ebf4e0c178664cb6913ed3698ef5c7ce'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:36.013404 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-93ba8488-0cbc-4479-b5b5-3c2dc2bb14bb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ebf4e0c178664cb6913ed3698ef5c7ce'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:36.014539 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-93ba8488-0cbc-4479-b5b5-3c2dc2bb14bb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ebf4e0c178664cb6913ed3698ef5c7ce'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:36.048740 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 934/1900] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Jul 27 10:48:35 2022] DELETE /identity/v3/users/ebf4e0c178664cb6913ed3698ef5c7ce => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:48:36.127602 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13e580b-525d-4ad3-b938-f95a9d09d4c5 admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:36.135453 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-b13e580b-525d-4ad3-b938-f95a9d09d4c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:36.176669 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13e580b-525d-4ad3-b938-f95a9d09d4c5 admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:36.325672 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ed3172-633f-4504-ad13-5d4ba9a5f171 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:36.326077 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ed3172-633f-4504-ad13-5d4ba9a5f171 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:36.326369 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ed3172-633f-4504-ad13-5d4ba9a5f171 admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:36.326670 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ed3172-633f-4504-ad13-5d4ba9a5f171 admin admin] PATH_INFO: `/v3/projects/3050f16606c44d039c13200dd1e3566b` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:36.332379 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ed3172-633f-4504-ad13-5d4ba9a5f171 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3050f16606c44d039c13200dd1e3566b)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:36.334037 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ed3172-633f-4504-ad13-5d4ba9a5f171 admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:36.446251 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-e9ed3172-633f-4504-ad13-5d4ba9a5f171 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3050f16606c44d039c13200dd1e3566b', 'request_id': 'req-e9ed3172-633f-4504-ad13-5d4ba9a5f171'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:36.463959 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-e9ed3172-633f-4504-ad13-5d4ba9a5f171 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3050f16606c44d039c13200dd1e3566b', 'request_id': 'req-e9ed3172-633f-4504-ad13-5d4ba9a5f171'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:36.473984 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-e9ed3172-633f-4504-ad13-5d4ba9a5f171 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3050f16606c44d039c13200dd1e3566b', 'request_id': 'req-e9ed3172-633f-4504-ad13-5d4ba9a5f171'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:36.509704 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-e9ed3172-633f-4504-ad13-5d4ba9a5f171 admin admin] The token cache is being invalidate because project 3050f16606c44d039c13200dd1e3566b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86827) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 10:48:36.509704 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-e9ed3172-633f-4504-ad13-5d4ba9a5f171 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:36.509704 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-e9ed3172-633f-4504-ad13-5d4ba9a5f171 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:36.509704 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 967/1901] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Jul 27 10:48:36 2022] DELETE /identity/v3/projects/3050f16606c44d039c13200dd1e3566b => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:48:36.510233 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0746f89f-1f8f-45e5-82df-afb07a1181be None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:36.510233 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0746f89f-1f8f-45e5-82df-afb07a1181be None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:36.510233 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0746f89f-1f8f-45e5-82df-afb07a1181be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:36.561512 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-0746f89f-1f8f-45e5-82df-afb07a1181be None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:48:36.562721 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-0746f89f-1f8f-45e5-82df-afb07a1181be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:36.677829 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 935/1902] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:48:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:48:36.678822 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ea45ba-6d1b-4601-a650-76b600c8477b None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:36.679118 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ea45ba-6d1b-4601-a650-76b600c8477b None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:36.679400 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ea45ba-6d1b-4601-a650-76b600c8477b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:36.724330 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-d9ea45ba-6d1b-4601-a650-76b600c8477b None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:48:36.725603 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-d9ea45ba-6d1b-4601-a650-76b600c8477b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:36.747734 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 968/1903] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:48:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:48:36.764783 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0746f89f-1f8f-45e5-82df-afb07a1181be None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:36.781467 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0746f89f-1f8f-45e5-82df-afb07a1181be None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:36.793567 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb04b86-643e-4074-aaa6-2b2f71fdfbf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:36.797777 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb04b86-643e-4074-aaa6-2b2f71fdfbf7 admin admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:36.798329 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb04b86-643e-4074-aaa6-2b2f71fdfbf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:36.798803 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb04b86-643e-4074-aaa6-2b2f71fdfbf7 admin admin] PATH_INFO: `/v3/domains` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:36.800132 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb04b86-643e-4074-aaa6-2b2f71fdfbf7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86828) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:48:36.801249 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb04b86-643e-4074-aaa6-2b2f71fdfbf7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:36.805183 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb04b86-643e-4074-aaa6-2b2f71fdfbf7 admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:36.838647 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 936/1904] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Jul 27 10:48:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:48:36.840334 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ea45ba-6d1b-4601-a650-76b600c8477b None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:36.860427 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ea45ba-6d1b-4601-a650-76b600c8477b None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:36.879345 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce454c7e-d62c-46d1-8a90-284cb0565264 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:36.880784 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce454c7e-d62c-46d1-8a90-284cb0565264 admin admin] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:36.882569 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce454c7e-d62c-46d1-8a90-284cb0565264 admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:36.882898 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce454c7e-d62c-46d1-8a90-284cb0565264 admin admin] PATH_INFO: `/v3/projects` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:36.884187 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce454c7e-d62c-46d1-8a90-284cb0565264 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:36.889146 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce454c7e-d62c-46d1-8a90-284cb0565264 admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:36.969935 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 969/1905] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Jul 27 10:48:36 2022] POST /identity/v3/projects => generated 381 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 10:48:36.974349 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb04b86-643e-4074-aaa6-2b2f71fdfbf7 admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:36.988441 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb04b86-643e-4074-aaa6-2b2f71fdfbf7 admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:37.107859 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9106212c-f8fc-4ee5-9930-a1e318b55919 None None] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:37.108790 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9106212c-f8fc-4ee5-9930-a1e318b55919 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:37.109479 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9106212c-f8fc-4ee5-9930-a1e318b55919 None None] PATH_INFO: `/` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:37.111426 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:48:37.112726 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 970/1906] 10.222.0.36 () {54 vars in 921 bytes} [Wed Jul 27 10:48:37 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 10:48:37.130638 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6239006-25df-4f73-a641-a0b065031f66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:37.131983 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6239006-25df-4f73-a641-a0b065031f66 admin admin] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:37.132930 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6239006-25df-4f73-a641-a0b065031f66 admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:37.133430 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6239006-25df-4f73-a641-a0b065031f66 admin admin] PATH_INFO: `/v3/users` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:37.134467 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6239006-25df-4f73-a641-a0b065031f66 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:37.142980 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6239006-25df-4f73-a641-a0b065031f66 admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:37.174266 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c04eeb-fa2a-4704-bc71-096217db5c69 None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:37.175626 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c04eeb-fa2a-4704-bc71-096217db5c69 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:37.176135 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c04eeb-fa2a-4704-bc71-096217db5c69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:37.179545 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 937/1907] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Jul 27 10:48:36 2022] POST /identity/v3/users => generated 291 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 10:48:37.186044 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6239006-25df-4f73-a641-a0b065031f66 admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:37.215746 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6239006-25df-4f73-a641-a0b065031f66 admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:37.229418 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e3f7df-efd2-41e5-aa4a-9333ff958fb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:37.229896 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e3f7df-efd2-41e5-aa4a-9333ff958fb9 admin admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:37.231055 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-66c04eeb-fa2a-4704-bc71-096217db5c69 None None] MFA Rules not processed for user `e059d8b6c28c4fcba925d05d7c98e4ac`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:48:37.232334 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-66c04eeb-fa2a-4704-bc71-096217db5c69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:37.234092 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e3f7df-efd2-41e5-aa4a-9333ff958fb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:37.234092 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e3f7df-efd2-41e5-aa4a-9333ff958fb9 admin admin] PATH_INFO: `/v3/roles` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:37.234953 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e3f7df-efd2-41e5-aa4a-9333ff958fb9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:37.236205 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e3f7df-efd2-41e5-aa4a-9333ff958fb9 admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:37.272947 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 938/1908] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 10:48:37 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 10:48:37.274245 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e3f7df-efd2-41e5-aa4a-9333ff958fb9 admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:37.305688 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e3f7df-efd2-41e5-aa4a-9333ff958fb9 admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:37.321755 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe326a9f-ff8b-4d15-9620-f1fc0fde58f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:37.325363 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe326a9f-ff8b-4d15-9620-f1fc0fde58f0 admin admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:37.325663 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe326a9f-ff8b-4d15-9620-f1fc0fde58f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:37.326025 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe326a9f-ff8b-4d15-9620-f1fc0fde58f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:37.327552 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe326a9f-ff8b-4d15-9620-f1fc0fde58f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:37.334528 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe326a9f-ff8b-4d15-9620-f1fc0fde58f0 admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:37.343119 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:48:37.366274 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 939/1909] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 10:48:37 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 10:48:37.367803 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe326a9f-ff8b-4d15-9620-f1fc0fde58f0 admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:37.369883 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-fe326a9f-ff8b-4d15-9620-f1fc0fde58f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:37.388267 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe326a9f-ff8b-4d15-9620-f1fc0fde58f0 admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:37.390291 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 971/1910] 10.222.0.36 () {58 vars in 1018 bytes} [Wed Jul 27 10:48:37 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (3 switches on core 0) Jul 27 10:48:37.391397 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c04eeb-fa2a-4704-bc71-096217db5c69 None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:37.407897 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c04eeb-fa2a-4704-bc71-096217db5c69 None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:37.413834 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4822f72-4445-4e35-b1c4-eea82ef58e47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:37.414948 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4822f72-4445-4e35-b1c4-eea82ef58e47 admin admin] REQUEST_METHOD: `PUT` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:37.415522 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4822f72-4445-4e35-b1c4-eea82ef58e47 admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:37.416743 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4822f72-4445-4e35-b1c4-eea82ef58e47 admin admin] PATH_INFO: `/v3/projects/37b26801bd514df6ae0d722db67cffeb/users/a9ff68648ee94cd48e9fcb0f3808b919/roles/4c9fc0b3baed441da2126be41cc6b04a` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:37.425142 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4396a4f0-79b1-486d-9911-be01661ab37f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:37.426765 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4396a4f0-79b1-486d-9911-be01661ab37f admin admin] REQUEST_METHOD: `DELETE` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:37.427146 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4396a4f0-79b1-486d-9911-be01661ab37f admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:37.427432 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4396a4f0-79b1-486d-9911-be01661ab37f admin admin] PATH_INFO: `/v3/projects/4d21e17346b343419d55a98ae4b9728f` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:37.428758 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4396a4f0-79b1-486d-9911-be01661ab37f admin admin] RBAC: Authorizing `identity:delete_project(project_id=4d21e17346b343419d55a98ae4b9728f)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:37.438986 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4396a4f0-79b1-486d-9911-be01661ab37f admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:37.445909 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4822f72-4445-4e35-b1c4-eea82ef58e47 admin admin] RBAC: Authorizing `identity:create_grant(project_id=37b26801bd514df6ae0d722db67cffeb, user_id=a9ff68648ee94cd48e9fcb0f3808b919, role_id=4c9fc0b3baed441da2126be41cc6b04a)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:37.449608 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4822f72-4445-4e35-b1c4-eea82ef58e47 admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:37.490649 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 972/1911] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 10:48:37 2022] PUT /identity/v3/projects/37b26801bd514df6ae0d722db67cffeb/users/a9ff68648ee94cd48e9fcb0f3808b919/roles/4c9fc0b3baed441da2126be41cc6b04a => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:48:37.490649 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4822f72-4445-4e35-b1c4-eea82ef58e47 admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:37.510497 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-f4822f72-4445-4e35-b1c4-eea82ef58e47 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:37.539143 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-4396a4f0-79b1-486d-9911-be01661ab37f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4d21e17346b343419d55a98ae4b9728f', 'request_id': 'req-4396a4f0-79b1-486d-9911-be01661ab37f'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:37.546266 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-4396a4f0-79b1-486d-9911-be01661ab37f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4d21e17346b343419d55a98ae4b9728f', 'request_id': 'req-4396a4f0-79b1-486d-9911-be01661ab37f'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:37.554668 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-4396a4f0-79b1-486d-9911-be01661ab37f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4d21e17346b343419d55a98ae4b9728f', 'request_id': 'req-4396a4f0-79b1-486d-9911-be01661ab37f'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:37.566448 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-4396a4f0-79b1-486d-9911-be01661ab37f admin admin] The token cache is being invalidate because project 4d21e17346b343419d55a98ae4b9728f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86828) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 10:48:37.566777 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-4396a4f0-79b1-486d-9911-be01661ab37f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:37.567480 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-4396a4f0-79b1-486d-9911-be01661ab37f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:37.569020 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 940/1912] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Jul 27 10:48:37 2022] DELETE /identity/v3/projects/4d21e17346b343419d55a98ae4b9728f => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:48:37.570422 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57bb9f0-514c-41ee-ad9f-efad13547ffe None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:37.570675 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57bb9f0-514c-41ee-ad9f-efad13547ffe None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:37.570948 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57bb9f0-514c-41ee-ad9f-efad13547ffe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:37.588330 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4822f72-4445-4e35-b1c4-eea82ef58e47 admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:37.599314 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-a57bb9f0-514c-41ee-ad9f-efad13547ffe None None] MFA Rules not processed for user `a9ff68648ee94cd48e9fcb0f3808b919`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:48:37.600557 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-a57bb9f0-514c-41ee-ad9f-efad13547ffe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:37.654092 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6dac34b-1246-4444-84ee-b80b0f4ef32b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e059d8b6c28c4fcba925d05d7c98e4ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:37.654830 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dac34b-1246-4444-84ee-b80b0f4ef32b service nova] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:37.655118 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dac34b-1246-4444-84ee-b80b0f4ef32b service nova] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:37.655462 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dac34b-1246-4444-84ee-b80b0f4ef32b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:37.657319 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-f6dac34b-1246-4444-84ee-b80b0f4ef32b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:37.668728 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 941/1913] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:48:37 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:48:37.669990 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3919ef5a-34e9-41e8-a05b-298a8550ef8e None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:37.670280 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3919ef5a-34e9-41e8-a05b-298a8550ef8e None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:37.670558 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3919ef5a-34e9-41e8-a05b-298a8550ef8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:37.675850 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6dac34b-1246-4444-84ee-b80b0f4ef32b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:37.677448 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6dac34b-1246-4444-84ee-b80b0f4ef32b service nova] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:37.688219 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 973/1914] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:48:37 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 10:48:37.689423 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-3919ef5a-34e9-41e8-a05b-298a8550ef8e None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:48:37.690658 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-3919ef5a-34e9-41e8-a05b-298a8550ef8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:37.691638 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6dac34b-1246-4444-84ee-b80b0f4ef32b service nova] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:37.692987 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-f6dac34b-1246-4444-84ee-b80b0f4ef32b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:37.726680 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:48:37.745284 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:48:37.756637 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 942/1915] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:48:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:48:37.765663 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df222d61-d37e-4e54-a155-188ebbb926c4 None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:37.765950 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df222d61-d37e-4e54-a155-188ebbb926c4 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:37.766248 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df222d61-d37e-4e54-a155-188ebbb926c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:37.771844 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6dac34b-1246-4444-84ee-b80b0f4ef32b service nova] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:37.777908 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0faa535e-9e66-4d7a-904b-bd6363bfb3a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:37.778542 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0faa535e-9e66-4d7a-904b-bd6363bfb3a5 admin admin] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:37.778996 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0faa535e-9e66-4d7a-904b-bd6363bfb3a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:37.779271 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0faa535e-9e66-4d7a-904b-bd6363bfb3a5 admin admin] PATH_INFO: `/v3/projects` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:37.779933 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0faa535e-9e66-4d7a-904b-bd6363bfb3a5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:37.781818 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0faa535e-9e66-4d7a-904b-bd6363bfb3a5 admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:37.785889 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-df222d61-d37e-4e54-a155-188ebbb926c4 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:48:37.787124 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-df222d61-d37e-4e54-a155-188ebbb926c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:37.795399 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 943/1916] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:48:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 10:48:37.799456 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df222d61-d37e-4e54-a155-188ebbb926c4 None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:37.808900 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df222d61-d37e-4e54-a155-188ebbb926c4 None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:37.811395 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 974/1917] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Jul 27 10:48:37 2022] POST /identity/v3/projects => generated 383 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 10:48:37.813550 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e21570-e5d4-4e6b-8868-6cf987f1c9db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:37.814164 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e21570-e5d4-4e6b-8868-6cf987f1c9db admin admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:37.814450 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e21570-e5d4-4e6b-8868-6cf987f1c9db admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:37.814851 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e21570-e5d4-4e6b-8868-6cf987f1c9db admin admin] PATH_INFO: `/v3/domains` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:37.815217 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0faa535e-9e66-4d7a-904b-bd6363bfb3a5 admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:37.815554 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e21570-e5d4-4e6b-8868-6cf987f1c9db admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86828) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:48:37.815857 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e21570-e5d4-4e6b-8868-6cf987f1c9db admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:37.817490 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e21570-e5d4-4e6b-8868-6cf987f1c9db admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:37.831012 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 944/1918] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Jul 27 10:48:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 10:48:37.835137 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e21570-e5d4-4e6b-8868-6cf987f1c9db admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:37.847840 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e21570-e5d4-4e6b-8868-6cf987f1c9db admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:37.886072 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0faa535e-9e66-4d7a-904b-bd6363bfb3a5 admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:37.891426 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaef433c-010b-4fa9-ae6f-6919f3c005d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:37.892055 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaef433c-010b-4fa9-ae6f-6919f3c005d7 admin admin] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:37.892328 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaef433c-010b-4fa9-ae6f-6919f3c005d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:37.892604 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaef433c-010b-4fa9-ae6f-6919f3c005d7 admin admin] PATH_INFO: `/v3/users` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:37.893407 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaef433c-010b-4fa9-ae6f-6919f3c005d7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:37.895377 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaef433c-010b-4fa9-ae6f-6919f3c005d7 admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:37.908832 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed853fd-6864-4c05-8ce9-021e058396e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:37.909566 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed853fd-6864-4c05-8ce9-021e058396e3 admin admin] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:37.909864 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed853fd-6864-4c05-8ce9-021e058396e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:37.910254 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed853fd-6864-4c05-8ce9-021e058396e3 admin admin] PATH_INFO: `/v3/projects` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:37.911019 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed853fd-6864-4c05-8ce9-021e058396e3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:37.912976 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed853fd-6864-4c05-8ce9-021e058396e3 admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:37.920930 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 975/1919] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Jul 27 10:48:37 2022] POST /identity/v3/users => generated 298 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 10:48:37.926194 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaef433c-010b-4fa9-ae6f-6919f3c005d7 admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:37.940332 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaef433c-010b-4fa9-ae6f-6919f3c005d7 admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:37.948196 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 945/1920] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Jul 27 10:48:37 2022] POST /identity/v3/projects => generated 375 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 10:48:37.949626 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607dd454-e5f1-491b-9e9b-89a4b91fe671 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:37.950137 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed853fd-6864-4c05-8ce9-021e058396e3 admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:37.951845 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607dd454-e5f1-491b-9e9b-89a4b91fe671 admin admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:37.952141 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607dd454-e5f1-491b-9e9b-89a4b91fe671 admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:37.952687 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607dd454-e5f1-491b-9e9b-89a4b91fe671 admin admin] PATH_INFO: `/v3/roles` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:37.953467 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607dd454-e5f1-491b-9e9b-89a4b91fe671 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:37.955165 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607dd454-e5f1-491b-9e9b-89a4b91fe671 admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:37.972252 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 976/1921] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 10:48:37 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 10:48:37.973197 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607dd454-e5f1-491b-9e9b-89a4b91fe671 admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:37.983536 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607dd454-e5f1-491b-9e9b-89a4b91fe671 admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:38.032836 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed853fd-6864-4c05-8ce9-021e058396e3 admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:38.038716 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27405483-efcc-448d-b8de-c513e03557ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:38.039548 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27405483-efcc-448d-b8de-c513e03557ab admin admin] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:38.039892 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27405483-efcc-448d-b8de-c513e03557ab admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:38.040244 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27405483-efcc-448d-b8de-c513e03557ab admin admin] PATH_INFO: `/v3/users` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:38.041595 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27405483-efcc-448d-b8de-c513e03557ab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:38.044038 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27405483-efcc-448d-b8de-c513e03557ab admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:38.062264 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e8a82d-1de4-45d2-9d1b-ac0e1742c21d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:38.063075 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e8a82d-1de4-45d2-9d1b-ac0e1742c21d admin admin] REQUEST_METHOD: `PUT` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:38.063370 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e8a82d-1de4-45d2-9d1b-ac0e1742c21d admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:38.063633 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e8a82d-1de4-45d2-9d1b-ac0e1742c21d admin admin] PATH_INFO: `/v3/projects/5254304966a9423e867fef253079bb47/users/ef39ebeec06a411dbd6f6176e71d2eb3/roles/21156560eb6b436bb567a40b4976f3f4` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:38.070944 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 946/1922] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Jul 27 10:48:37 2022] POST /identity/v3/users => generated 288 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 10:48:38.075416 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27405483-efcc-448d-b8de-c513e03557ab admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:38.083048 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e8a82d-1de4-45d2-9d1b-ac0e1742c21d admin admin] RBAC: Authorizing `identity:create_grant(project_id=5254304966a9423e867fef253079bb47, user_id=ef39ebeec06a411dbd6f6176e71d2eb3, role_id=21156560eb6b436bb567a40b4976f3f4)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:38.085791 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e8a82d-1de4-45d2-9d1b-ac0e1742c21d admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:38.086040 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27405483-efcc-448d-b8de-c513e03557ab admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:38.092352 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938521e7-1b6a-4d73-887f-e999b526255c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:38.093012 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938521e7-1b6a-4d73-887f-e999b526255c admin admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:38.093293 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938521e7-1b6a-4d73-887f-e999b526255c admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:38.093569 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938521e7-1b6a-4d73-887f-e999b526255c admin admin] PATH_INFO: `/v3/roles` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:38.094272 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938521e7-1b6a-4d73-887f-e999b526255c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:38.095917 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938521e7-1b6a-4d73-887f-e999b526255c admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:38.100285 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 977/1923] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 10:48:37 2022] PUT /identity/v3/projects/5254304966a9423e867fef253079bb47/users/ef39ebeec06a411dbd6f6176e71d2eb3/roles/21156560eb6b436bb567a40b4976f3f4 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:48:38.104887 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e8a82d-1de4-45d2-9d1b-ac0e1742c21d admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:38.111297 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 947/1924] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 10:48:38 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 10:48:38.115636 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938521e7-1b6a-4d73-887f-e999b526255c admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:38.130616 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938521e7-1b6a-4d73-887f-e999b526255c admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:38.144802 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:48:38.152438 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:48:38.180605 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e8a82d-1de4-45d2-9d1b-ac0e1742c21d admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:38.186079 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0d286c-27bd-4e2d-8428-6e1364bb1d2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:38.186851 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0d286c-27bd-4e2d-8428-6e1364bb1d2b admin admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:38.187133 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0d286c-27bd-4e2d-8428-6e1364bb1d2b admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:38.187413 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0d286c-27bd-4e2d-8428-6e1364bb1d2b admin admin] PATH_INFO: `/v3/roles` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:38.188114 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0d286c-27bd-4e2d-8428-6e1364bb1d2b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:38.189764 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0d286c-27bd-4e2d-8428-6e1364bb1d2b admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:38.191818 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bbdc11-b6b5-4745-955e-ce8f10562788 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:38.192436 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bbdc11-b6b5-4745-955e-ce8f10562788 admin admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:38.192714 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bbdc11-b6b5-4745-955e-ce8f10562788 admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:38.192986 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bbdc11-b6b5-4745-955e-ce8f10562788 admin admin] PATH_INFO: `/v3/roles` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:38.193674 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5bbdc11-b6b5-4745-955e-ce8f10562788 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:38.195351 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5bbdc11-b6b5-4745-955e-ce8f10562788 admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:38.200443 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 978/1925] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 10:48:38 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 10:48:38.204545 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0d286c-27bd-4e2d-8428-6e1364bb1d2b admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:38.206433 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 948/1926] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 10:48:38 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 10:48:38.210244 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bbdc11-b6b5-4745-955e-ce8f10562788 admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:38.214827 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0d286c-27bd-4e2d-8428-6e1364bb1d2b admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:38.220019 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bbdc11-b6b5-4745-955e-ce8f10562788 admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:38.220651 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb04edca-4a3f-4bb6-ab5f-78b2829b71bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:38.221412 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb04edca-4a3f-4bb6-ab5f-78b2829b71bb admin admin] REQUEST_METHOD: `PUT` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:38.221654 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb04edca-4a3f-4bb6-ab5f-78b2829b71bb admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:38.221938 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb04edca-4a3f-4bb6-ab5f-78b2829b71bb admin admin] PATH_INFO: `/v3/projects/5254304966a9423e867fef253079bb47/users/ef39ebeec06a411dbd6f6176e71d2eb3/roles/21156560eb6b436bb567a40b4976f3f4` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:38.223998 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb04edca-4a3f-4bb6-ab5f-78b2829b71bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=5254304966a9423e867fef253079bb47, user_id=ef39ebeec06a411dbd6f6176e71d2eb3, role_id=21156560eb6b436bb567a40b4976f3f4)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:38.225076 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360dacde-e830-4730-9496-e0d227e53650 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:38.225732 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360dacde-e830-4730-9496-e0d227e53650 admin admin] REQUEST_METHOD: `PUT` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:38.226012 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360dacde-e830-4730-9496-e0d227e53650 admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:38.226286 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360dacde-e830-4730-9496-e0d227e53650 admin admin] PATH_INFO: `/v3/projects/1ea4ccfa264a47499fa5bdcee98c046e/users/40cbaea1548543d69ea24ece1507fac4/roles/4c9fc0b3baed441da2126be41cc6b04a` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:38.226673 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb04edca-4a3f-4bb6-ab5f-78b2829b71bb admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:38.240188 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 979/1927] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 10:48:38 2022] PUT /identity/v3/projects/5254304966a9423e867fef253079bb47/users/ef39ebeec06a411dbd6f6176e71d2eb3/roles/21156560eb6b436bb567a40b4976f3f4 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 10:48:38.241797 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360dacde-e830-4730-9496-e0d227e53650 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1ea4ccfa264a47499fa5bdcee98c046e, user_id=40cbaea1548543d69ea24ece1507fac4, role_id=4c9fc0b3baed441da2126be41cc6b04a)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:38.244638 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360dacde-e830-4730-9496-e0d227e53650 admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:38.251502 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af63dd0-47d4-4982-8d89-96bf587abb86 None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:38.252042 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af63dd0-47d4-4982-8d89-96bf587abb86 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:38.252392 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af63dd0-47d4-4982-8d89-96bf587abb86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:38.266048 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 949/1928] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 10:48:38 2022] PUT /identity/v3/projects/1ea4ccfa264a47499fa5bdcee98c046e/users/40cbaea1548543d69ea24ece1507fac4/roles/4c9fc0b3baed441da2126be41cc6b04a => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 10:48:38.278191 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d9860c-fced-42cc-9b0e-6cac2d6d748f None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:38.278769 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d9860c-fced-42cc-9b0e-6cac2d6d748f None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:38.278990 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d9860c-fced-42cc-9b0e-6cac2d6d748f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:38.290494 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-0af63dd0-47d4-4982-8d89-96bf587abb86 None None] MFA Rules not processed for user `ef39ebeec06a411dbd6f6176e71d2eb3`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:48:38.291735 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-0af63dd0-47d4-4982-8d89-96bf587abb86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:38.309294 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-11d9860c-fced-42cc-9b0e-6cac2d6d748f None None] MFA Rules not processed for user `40cbaea1548543d69ea24ece1507fac4`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:48:38.311216 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-11d9860c-fced-42cc-9b0e-6cac2d6d748f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:38.364357 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 980/1929] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:48:38 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 10:48:38.391441 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 950/1930] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:48:38 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 10:48:38.394698 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af63dd0-47d4-4982-8d89-96bf587abb86 None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:38.395499 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d9860c-fced-42cc-9b0e-6cac2d6d748f None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:38.396262 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-0af63dd0-47d4-4982-8d89-96bf587abb86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:38.462638 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af63dd0-47d4-4982-8d89-96bf587abb86 None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:38.467307 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965a958f-90c5-4b96-86be-ab55be510ad5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '864d35f268cf45148756109759bb2743', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:38.468055 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965a958f-90c5-4b96-86be-ab55be510ad5 service cinder] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:38.468335 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965a958f-90c5-4b96-86be-ab55be510ad5 service cinder] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:38.468618 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965a958f-90c5-4b96-86be-ab55be510ad5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:38.476077 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d9860c-fced-42cc-9b0e-6cac2d6d748f None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:38.479726 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965a958f-90c5-4b96-86be-ab55be510ad5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:38.481877 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0686af85-d156-47ec-beb0-27c95c3e4bd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:38.482032 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965a958f-90c5-4b96-86be-ab55be510ad5 service cinder] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:38.482675 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0686af85-d156-47ec-beb0-27c95c3e4bd4 admin admin] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:38.482969 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0686af85-d156-47ec-beb0-27c95c3e4bd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:38.483248 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0686af85-d156-47ec-beb0-27c95c3e4bd4 admin admin] PATH_INFO: `/v3/projects` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:38.483922 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0686af85-d156-47ec-beb0-27c95c3e4bd4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:38.485877 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0686af85-d156-47ec-beb0-27c95c3e4bd4 admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:38.494069 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 981/1931] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:48:38 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 10:48:38.527388 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 951/1932] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Jul 27 10:48:38 2022] POST /identity/v3/projects => generated 373 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 10:48:38.529642 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965a958f-90c5-4b96-86be-ab55be510ad5 service cinder] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:38.545407 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965a958f-90c5-4b96-86be-ab55be510ad5 service cinder] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:38.598168 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:48:38.629266 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44380566-7628-4b66-bb2f-889d3b1effdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:38.629901 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44380566-7628-4b66-bb2f-889d3b1effdc admin admin] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:38.630179 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44380566-7628-4b66-bb2f-889d3b1effdc admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:38.630463 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44380566-7628-4b66-bb2f-889d3b1effdc admin admin] PATH_INFO: `/v3/users` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:38.631171 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44380566-7628-4b66-bb2f-889d3b1effdc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:38.633147 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44380566-7628-4b66-bb2f-889d3b1effdc admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:38.664694 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 982/1933] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Jul 27 10:48:38 2022] POST /identity/v3/users => generated 293 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 10:48:38.665239 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0686af85-d156-47ec-beb0-27c95c3e4bd4 admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:38.677908 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0686af85-d156-47ec-beb0-27c95c3e4bd4 admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:38.682674 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44380566-7628-4b66-bb2f-889d3b1effdc admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:38.683153 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4620ce-f53e-4be4-aecb-932cdaf10a06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:38.684321 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4620ce-f53e-4be4-aecb-932cdaf10a06 admin admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:38.684608 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4620ce-f53e-4be4-aecb-932cdaf10a06 admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:38.685141 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4620ce-f53e-4be4-aecb-932cdaf10a06 admin admin] PATH_INFO: `/v3/roles` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:38.686413 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4620ce-f53e-4be4-aecb-932cdaf10a06 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:38.687556 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4620ce-f53e-4be4-aecb-932cdaf10a06 admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:38.690771 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:48:38.701859 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 952/1934] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 10:48:38 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 10:48:38.706104 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4620ce-f53e-4be4-aecb-932cdaf10a06 admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:38.720196 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4620ce-f53e-4be4-aecb-932cdaf10a06 admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:38.724899 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ac163b-b96d-4f35-a5d8-90cdc038b81b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:38.725570 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ac163b-b96d-4f35-a5d8-90cdc038b81b admin admin] REQUEST_METHOD: `PUT` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:38.725859 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ac163b-b96d-4f35-a5d8-90cdc038b81b admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:38.726138 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ac163b-b96d-4f35-a5d8-90cdc038b81b admin admin] PATH_INFO: `/v3/projects/50958e3a173847db998537e22c8f3efb/users/b3f90d3b43b14140ade93463d368c963/roles/21156560eb6b436bb567a40b4976f3f4` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:38.741751 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ac163b-b96d-4f35-a5d8-90cdc038b81b admin admin] RBAC: Authorizing `identity:create_grant(project_id=50958e3a173847db998537e22c8f3efb, user_id=b3f90d3b43b14140ade93463d368c963, role_id=21156560eb6b436bb567a40b4976f3f4)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:38.744492 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ac163b-b96d-4f35-a5d8-90cdc038b81b admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:38.748171 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44380566-7628-4b66-bb2f-889d3b1effdc admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:38.752744 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664356fb-bfe5-43fb-a306-0d7aea5bebdf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '864d35f268cf45148756109759bb2743', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:38.753520 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664356fb-bfe5-43fb-a306-0d7aea5bebdf service cinder] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:38.753809 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664356fb-bfe5-43fb-a306-0d7aea5bebdf service cinder] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:38.754074 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664356fb-bfe5-43fb-a306-0d7aea5bebdf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:38.758812 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 953/1935] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 10:48:38 2022] PUT /identity/v3/projects/50958e3a173847db998537e22c8f3efb/users/b3f90d3b43b14140ade93463d368c963/roles/21156560eb6b436bb567a40b4976f3f4 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 10:48:38.762814 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ac163b-b96d-4f35-a5d8-90cdc038b81b admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:38.813259 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664356fb-bfe5-43fb-a306-0d7aea5bebdf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:38.814749 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664356fb-bfe5-43fb-a306-0d7aea5bebdf service cinder] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:38.823898 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 983/1936] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:48:38 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 10:48:38.825644 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ac163b-b96d-4f35-a5d8-90cdc038b81b admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:38.830865 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76053c46-abd8-451d-9a9b-316b5e30ba58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:38.831489 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76053c46-abd8-451d-9a9b-316b5e30ba58 admin admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:38.831769 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76053c46-abd8-451d-9a9b-316b5e30ba58 admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:38.832049 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76053c46-abd8-451d-9a9b-316b5e30ba58 admin admin] PATH_INFO: `/v3/roles` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:38.832741 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76053c46-abd8-451d-9a9b-316b5e30ba58 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:38.834353 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76053c46-abd8-451d-9a9b-316b5e30ba58 admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:38.844963 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 954/1937] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 10:48:38 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 10:48:38.849280 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664356fb-bfe5-43fb-a306-0d7aea5bebdf service cinder] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:38.860751 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664356fb-bfe5-43fb-a306-0d7aea5bebdf service cinder] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:38.865342 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db02b4a6-705c-475d-9520-2be4b38961f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:38.866530 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db02b4a6-705c-475d-9520-2be4b38961f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:38.866530 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db02b4a6-705c-475d-9520-2be4b38961f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:38.866530 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db02b4a6-705c-475d-9520-2be4b38961f1 admin admin] PATH_INFO: `/v3/projects/50958e3a173847db998537e22c8f3efb/users/b3f90d3b43b14140ade93463d368c963/roles/21156560eb6b436bb567a40b4976f3f4` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:38.868643 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db02b4a6-705c-475d-9520-2be4b38961f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=50958e3a173847db998537e22c8f3efb, user_id=b3f90d3b43b14140ade93463d368c963, role_id=21156560eb6b436bb567a40b4976f3f4)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:38.873159 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db02b4a6-705c-475d-9520-2be4b38961f1 admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:38.885374 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 984/1938] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 10:48:38 2022] PUT /identity/v3/projects/50958e3a173847db998537e22c8f3efb/users/b3f90d3b43b14140ade93463d368c963/roles/21156560eb6b436bb567a40b4976f3f4 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:48:38.895511 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2795d51f-198f-409a-9108-5ea4cea96383 None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:38.895984 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2795d51f-198f-409a-9108-5ea4cea96383 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:38.896074 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2795d51f-198f-409a-9108-5ea4cea96383 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:38.925477 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-2795d51f-198f-409a-9108-5ea4cea96383 None None] MFA Rules not processed for user `b3f90d3b43b14140ade93463d368c963`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:48:38.926726 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-2795d51f-198f-409a-9108-5ea4cea96383 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:39.006437 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 955/1939] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:48:38 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 10:48:39.018295 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db02b4a6-705c-475d-9520-2be4b38961f1 admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:39.020259 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-db02b4a6-705c-475d-9520-2be4b38961f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:39.086416 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db02b4a6-705c-475d-9520-2be4b38961f1 admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:39.090629 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5bf28b-ba0f-4dd9-8c13-3a5afae5ba66 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '864d35f268cf45148756109759bb2743', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:39.091307 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5bf28b-ba0f-4dd9-8c13-3a5afae5ba66 service cinder] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:39.091555 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5bf28b-ba0f-4dd9-8c13-3a5afae5ba66 service cinder] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:39.091835 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5bf28b-ba0f-4dd9-8c13-3a5afae5ba66 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:39.119465 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:48:39.182299 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5bf28b-ba0f-4dd9-8c13-3a5afae5ba66 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:39.184494 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5bf28b-ba0f-4dd9-8c13-3a5afae5ba66 service cinder] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:39.198104 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 985/1940] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:48:39 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 10:48:41.962384 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2795d51f-198f-409a-9108-5ea4cea96383 None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:41.964187 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-2795d51f-198f-409a-9108-5ea4cea96383 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:42.033449 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:48:42.039303 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2795d51f-198f-409a-9108-5ea4cea96383 None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:42.045243 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6961133f-e9ba-4860-9bfb-89c75edc8610 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:42.045914 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6961133f-e9ba-4860-9bfb-89c75edc8610 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:42.046200 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6961133f-e9ba-4860-9bfb-89c75edc8610 admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:42.046493 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6961133f-e9ba-4860-9bfb-89c75edc8610 admin admin] PATH_INFO: `/v3/users/fa647284289e40ecae75fb2b8c44b5cb` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:42.047687 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6961133f-e9ba-4860-9bfb-89c75edc8610 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fa647284289e40ecae75fb2b8c44b5cb)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:42.049631 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6961133f-e9ba-4860-9bfb-89c75edc8610 admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:42.088901 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5bf28b-ba0f-4dd9-8c13-3a5afae5ba66 service cinder] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:42.090864 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-8c5bf28b-ba0f-4dd9-8c13-3a5afae5ba66 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:42.101490 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-6961133f-e9ba-4860-9bfb-89c75edc8610 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fa647284289e40ecae75fb2b8c44b5cb'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:42.105968 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5bf28b-ba0f-4dd9-8c13-3a5afae5ba66 service cinder] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:42.109072 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-6961133f-e9ba-4860-9bfb-89c75edc8610 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fa647284289e40ecae75fb2b8c44b5cb'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:42.110895 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3fd10b-d118-44e1-8bbe-e124c2b7f03f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:42.111545 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3fd10b-d118-44e1-8bbe-e124c2b7f03f admin admin] REQUEST_METHOD: `DELETE` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:42.111823 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3fd10b-d118-44e1-8bbe-e124c2b7f03f admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:42.112132 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3fd10b-d118-44e1-8bbe-e124c2b7f03f admin admin] PATH_INFO: `/v3/users/5194030f71bc49ca94932bfe21f1db56` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:42.113415 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3fd10b-d118-44e1-8bbe-e124c2b7f03f admin admin] RBAC: Authorizing `identity:delete_user(user_id=5194030f71bc49ca94932bfe21f1db56)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:42.116069 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3fd10b-d118-44e1-8bbe-e124c2b7f03f admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:42.116900 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-6961133f-e9ba-4860-9bfb-89c75edc8610 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fa647284289e40ecae75fb2b8c44b5cb'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:42.117543 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-6961133f-e9ba-4860-9bfb-89c75edc8610 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fa647284289e40ecae75fb2b8c44b5cb'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:42.118163 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-6961133f-e9ba-4860-9bfb-89c75edc8610 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fa647284289e40ecae75fb2b8c44b5cb'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:42.139069 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 956/1941] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Jul 27 10:48:41 2022] DELETE /identity/v3/users/fa647284289e40ecae75fb2b8c44b5cb => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:48:42.143310 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6caf6f-c7f3-4109-ba0c-df96ca9ceff1 None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:42.143603 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6caf6f-c7f3-4109-ba0c-df96ca9ceff1 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:42.143882 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6caf6f-c7f3-4109-ba0c-df96ca9ceff1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:42.166840 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-eb6caf6f-c7f3-4109-ba0c-df96ca9ceff1 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:48:42.168077 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-eb6caf6f-c7f3-4109-ba0c-df96ca9ceff1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:42.171377 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-be3fd10b-d118-44e1-8bbe-e124c2b7f03f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5194030f71bc49ca94932bfe21f1db56'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:42.180120 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-be3fd10b-d118-44e1-8bbe-e124c2b7f03f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5194030f71bc49ca94932bfe21f1db56'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:42.188896 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-be3fd10b-d118-44e1-8bbe-e124c2b7f03f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5194030f71bc49ca94932bfe21f1db56'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:42.189637 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-be3fd10b-d118-44e1-8bbe-e124c2b7f03f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5194030f71bc49ca94932bfe21f1db56'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:42.190243 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-be3fd10b-d118-44e1-8bbe-e124c2b7f03f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5194030f71bc49ca94932bfe21f1db56'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:42.210402 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 986/1942] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Jul 27 10:48:42 2022] DELETE /identity/v3/users/5194030f71bc49ca94932bfe21f1db56 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:48:42.214883 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bbb6f6-4b54-4e93-a072-ffc9eaf197df None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:42.215109 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bbb6f6-4b54-4e93-a072-ffc9eaf197df None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:42.215387 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bbb6f6-4b54-4e93-a072-ffc9eaf197df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:42.232865 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 957/1943] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:48:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 10:48:42.234423 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-07bbb6f6-4b54-4e93-a072-ffc9eaf197df None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:48:42.235719 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-07bbb6f6-4b54-4e93-a072-ffc9eaf197df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:42.244530 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6caf6f-c7f3-4109-ba0c-df96ca9ceff1 None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:42.246002 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-eb6caf6f-c7f3-4109-ba0c-df96ca9ceff1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:42.299349 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 987/1944] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:48:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 10:48:42.305289 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6caf6f-c7f3-4109-ba0c-df96ca9ceff1 None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:42.309158 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bbb6f6-4b54-4e93-a072-ffc9eaf197df None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:42.334723 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:48:42.346536 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:48:42.358998 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a11a9c-a504-4075-b5b2-7ab7db506f83 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7922d0a1a7c4e808d0b2a80dca0da40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:42.359627 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a11a9c-a504-4075-b5b2-7ab7db506f83 service neutron] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:42.360018 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a11a9c-a504-4075-b5b2-7ab7db506f83 service neutron] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:42.360297 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a11a9c-a504-4075-b5b2-7ab7db506f83 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:42.368073 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bbb6f6-4b54-4e93-a072-ffc9eaf197df None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:42.369497 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a11a9c-a504-4075-b5b2-7ab7db506f83 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:42.371669 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a11a9c-a504-4075-b5b2-7ab7db506f83 service neutron] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:42.372794 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef31e19-e25b-4a43-87e4-605e1db4e0ee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7922d0a1a7c4e808d0b2a80dca0da40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:42.373452 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef31e19-e25b-4a43-87e4-605e1db4e0ee service neutron] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:42.373859 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef31e19-e25b-4a43-87e4-605e1db4e0ee service neutron] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:42.374771 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef31e19-e25b-4a43-87e4-605e1db4e0ee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:42.383026 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 958/1945] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:48:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 10:48:42.398913 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef31e19-e25b-4a43-87e4-605e1db4e0ee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:42.401687 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef31e19-e25b-4a43-87e4-605e1db4e0ee service neutron] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:42.422392 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 988/1946] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:48:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 10:48:42.572185 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a11a9c-a504-4075-b5b2-7ab7db506f83 service neutron] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:42.574160 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-01a11a9c-a504-4075-b5b2-7ab7db506f83 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:42.603303 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a11a9c-a504-4075-b5b2-7ab7db506f83 service neutron] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:42.621108 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60acf64f-5437-4cd1-84c2-262ab05b7dad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:42.622008 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60acf64f-5437-4cd1-84c2-262ab05b7dad admin admin] REQUEST_METHOD: `DELETE` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:42.622295 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60acf64f-5437-4cd1-84c2-262ab05b7dad admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:42.622569 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60acf64f-5437-4cd1-84c2-262ab05b7dad admin admin] PATH_INFO: `/v3/users/a9ff68648ee94cd48e9fcb0f3808b919` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:42.626334 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60acf64f-5437-4cd1-84c2-262ab05b7dad admin admin] RBAC: Authorizing `identity:delete_user(user_id=a9ff68648ee94cd48e9fcb0f3808b919)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:42.628356 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60acf64f-5437-4cd1-84c2-262ab05b7dad admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:42.721507 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-60acf64f-5437-4cd1-84c2-262ab05b7dad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a9ff68648ee94cd48e9fcb0f3808b919'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:42.735944 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-60acf64f-5437-4cd1-84c2-262ab05b7dad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a9ff68648ee94cd48e9fcb0f3808b919'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:42.766315 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-60acf64f-5437-4cd1-84c2-262ab05b7dad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a9ff68648ee94cd48e9fcb0f3808b919'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:42.767170 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-60acf64f-5437-4cd1-84c2-262ab05b7dad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a9ff68648ee94cd48e9fcb0f3808b919'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:42.768027 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-60acf64f-5437-4cd1-84c2-262ab05b7dad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a9ff68648ee94cd48e9fcb0f3808b919'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:42.827342 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 959/1947] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Jul 27 10:48:42 2022] DELETE /identity/v3/users/a9ff68648ee94cd48e9fcb0f3808b919 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:48:42.832459 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23e477f-a4b0-48e8-986f-34594c9c7f2a None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:42.832798 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23e477f-a4b0-48e8-986f-34594c9c7f2a None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:42.836744 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23e477f-a4b0-48e8-986f-34594c9c7f2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:42.864892 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-a23e477f-a4b0-48e8-986f-34594c9c7f2a None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:48:42.866339 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-a23e477f-a4b0-48e8-986f-34594c9c7f2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:42.998862 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 989/1948] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:48:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:48:43.023017 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60acf64f-5437-4cd1-84c2-262ab05b7dad admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:43.026088 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-60acf64f-5437-4cd1-84c2-262ab05b7dad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:43.043194 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23e477f-a4b0-48e8-986f-34594c9c7f2a None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:43.045359 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-a23e477f-a4b0-48e8-986f-34594c9c7f2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:43.080945 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23e477f-a4b0-48e8-986f-34594c9c7f2a None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:43.088472 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb8c3f2-3fab-45c6-a8db-634a8bdc6030 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:43.089116 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb8c3f2-3fab-45c6-a8db-634a8bdc6030 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:43.089733 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb8c3f2-3fab-45c6-a8db-634a8bdc6030 admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:43.090312 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb8c3f2-3fab-45c6-a8db-634a8bdc6030 admin admin] PATH_INFO: `/v3/projects/f9dbc435a6354c1a8a3d16234c20b5db` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:43.092699 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb8c3f2-3fab-45c6-a8db-634a8bdc6030 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f9dbc435a6354c1a8a3d16234c20b5db)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:43.095272 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb8c3f2-3fab-45c6-a8db-634a8bdc6030 admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:43.127871 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60acf64f-5437-4cd1-84c2-262ab05b7dad admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:43.132126 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3d0531-6ac5-495d-9a93-f48cca4a3741 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7922d0a1a7c4e808d0b2a80dca0da40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:43.132571 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3d0531-6ac5-495d-9a93-f48cca4a3741 service neutron] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:43.133012 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3d0531-6ac5-495d-9a93-f48cca4a3741 service neutron] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:43.133281 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3d0531-6ac5-495d-9a93-f48cca4a3741 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:43.146337 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3d0531-6ac5-495d-9a93-f48cca4a3741 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:43.147714 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3d0531-6ac5-495d-9a93-f48cca4a3741 service neutron] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:43.158748 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 960/1949] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:48:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 10:48:43.159589 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3d0531-6ac5-495d-9a93-f48cca4a3741 service neutron] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:43.160938 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-1d3d0531-6ac5-495d-9a93-f48cca4a3741 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:43.181661 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-2bb8c3f2-3fab-45c6-a8db-634a8bdc6030 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f9dbc435a6354c1a8a3d16234c20b5db', 'request_id': 'req-2bb8c3f2-3fab-45c6-a8db-634a8bdc6030'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:43.187027 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3d0531-6ac5-495d-9a93-f48cca4a3741 service neutron] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:43.191160 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-2bb8c3f2-3fab-45c6-a8db-634a8bdc6030 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f9dbc435a6354c1a8a3d16234c20b5db', 'request_id': 'req-2bb8c3f2-3fab-45c6-a8db-634a8bdc6030'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:43.205257 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-2bb8c3f2-3fab-45c6-a8db-634a8bdc6030 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f9dbc435a6354c1a8a3d16234c20b5db', 'request_id': 'req-2bb8c3f2-3fab-45c6-a8db-634a8bdc6030'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:43.241594 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-2bb8c3f2-3fab-45c6-a8db-634a8bdc6030 admin admin] The token cache is being invalidate because project f9dbc435a6354c1a8a3d16234c20b5db was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86827) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 10:48:43.241901 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-2bb8c3f2-3fab-45c6-a8db-634a8bdc6030 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:43.243734 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-2bb8c3f2-3fab-45c6-a8db-634a8bdc6030 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:43.252003 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 990/1950] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Jul 27 10:48:43 2022] DELETE /identity/v3/projects/f9dbc435a6354c1a8a3d16234c20b5db => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 10:48:43.252969 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb8c3f2-3fab-45c6-a8db-634a8bdc6030 admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:43.255493 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-2bb8c3f2-3fab-45c6-a8db-634a8bdc6030 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:43.299007 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:48:43.330976 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91acfbdf-9126-46d8-9629-04d39101b503 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:43.331654 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91acfbdf-9126-46d8-9629-04d39101b503 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:43.331941 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91acfbdf-9126-46d8-9629-04d39101b503 admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:43.332221 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91acfbdf-9126-46d8-9629-04d39101b503 admin admin] PATH_INFO: `/v3/users/40cbaea1548543d69ea24ece1507fac4` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:43.334675 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91acfbdf-9126-46d8-9629-04d39101b503 admin admin] RBAC: Authorizing `identity:delete_user(user_id=40cbaea1548543d69ea24ece1507fac4)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:43.336791 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91acfbdf-9126-46d8-9629-04d39101b503 admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:43.415781 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-91acfbdf-9126-46d8-9629-04d39101b503 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '40cbaea1548543d69ea24ece1507fac4'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:43.425525 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-91acfbdf-9126-46d8-9629-04d39101b503 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '40cbaea1548543d69ea24ece1507fac4'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:43.429031 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb8c3f2-3fab-45c6-a8db-634a8bdc6030 admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:43.435379 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-91acfbdf-9126-46d8-9629-04d39101b503 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '40cbaea1548543d69ea24ece1507fac4'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:43.436182 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a41fd7-8347-43ea-a470-fa43d868bc70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:43.436381 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-91acfbdf-9126-46d8-9629-04d39101b503 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '40cbaea1548543d69ea24ece1507fac4'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:43.436972 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a41fd7-8347-43ea-a470-fa43d868bc70 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:43.437341 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a41fd7-8347-43ea-a470-fa43d868bc70 admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:43.437506 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-91acfbdf-9126-46d8-9629-04d39101b503 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '40cbaea1548543d69ea24ece1507fac4'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:43.437799 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a41fd7-8347-43ea-a470-fa43d868bc70 admin admin] PATH_INFO: `/v3/projects/d571ca9b647f483a9663b6cf4872816a` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:43.440157 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a41fd7-8347-43ea-a470-fa43d868bc70 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d571ca9b647f483a9663b6cf4872816a)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:43.442390 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a41fd7-8347-43ea-a470-fa43d868bc70 admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:43.445766 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:48:43.479690 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 961/1951] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Jul 27 10:48:43 2022] DELETE /identity/v3/users/40cbaea1548543d69ea24ece1507fac4 => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:48:43.484244 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91acfbdf-9126-46d8-9629-04d39101b503 admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:43.489203 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-91acfbdf-9126-46d8-9629-04d39101b503 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:43.545055 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-05a41fd7-8347-43ea-a470-fa43d868bc70 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd571ca9b647f483a9663b6cf4872816a', 'request_id': 'req-05a41fd7-8347-43ea-a470-fa43d868bc70'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:43.568010 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-05a41fd7-8347-43ea-a470-fa43d868bc70 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd571ca9b647f483a9663b6cf4872816a', 'request_id': 'req-05a41fd7-8347-43ea-a470-fa43d868bc70'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:43.580037 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-05a41fd7-8347-43ea-a470-fa43d868bc70 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd571ca9b647f483a9663b6cf4872816a', 'request_id': 'req-05a41fd7-8347-43ea-a470-fa43d868bc70'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:43.596177 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91acfbdf-9126-46d8-9629-04d39101b503 admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:43.605267 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-05a41fd7-8347-43ea-a470-fa43d868bc70 admin admin] The token cache is being invalidate because project d571ca9b647f483a9663b6cf4872816a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86827) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 10:48:43.606171 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-05a41fd7-8347-43ea-a470-fa43d868bc70 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:43.607048 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-05a41fd7-8347-43ea-a470-fa43d868bc70 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:43.613832 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 991/1952] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Jul 27 10:48:43 2022] DELETE /identity/v3/projects/d571ca9b647f483a9663b6cf4872816a => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:48:43.615196 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9da843-fd05-4399-b7b1-5995332414cb None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:43.615589 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9da843-fd05-4399-b7b1-5995332414cb None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:43.615989 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9da843-fd05-4399-b7b1-5995332414cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:43.643216 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-3e9da843-fd05-4399-b7b1-5995332414cb None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:48:43.646447 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-3e9da843-fd05-4399-b7b1-5995332414cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:43.693001 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e1c8cf-24d5-4235-9c72-3bf7392b3c38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:43.693613 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e1c8cf-24d5-4235-9c72-3bf7392b3c38 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:43.694800 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e1c8cf-24d5-4235-9c72-3bf7392b3c38 admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:43.695087 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e1c8cf-24d5-4235-9c72-3bf7392b3c38 admin admin] PATH_INFO: `/v3/users/838adbfb1b7b4b35bfd49dde18d5d9e4` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:43.696309 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e1c8cf-24d5-4235-9c72-3bf7392b3c38 admin admin] RBAC: Authorizing `identity:delete_user(user_id=838adbfb1b7b4b35bfd49dde18d5d9e4)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:43.699432 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e1c8cf-24d5-4235-9c72-3bf7392b3c38 admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:43.718416 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:48:43.736338 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 992/1953] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:48:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:48:43.737533 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9da843-fd05-4399-b7b1-5995332414cb None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:43.739607 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-3e9da843-fd05-4399-b7b1-5995332414cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:43.752920 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9da843-fd05-4399-b7b1-5995332414cb None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:43.757312 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0404efc-3fe1-4c24-b2b4-e6d1a159f4ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:43.758488 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0404efc-3fe1-4c24-b2b4-e6d1a159f4ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:43.758488 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0404efc-3fe1-4c24-b2b4-e6d1a159f4ce admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:43.758488 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0404efc-3fe1-4c24-b2b4-e6d1a159f4ce admin admin] PATH_INFO: `/v3/users/c38f9d253f024686ba765c6a7fec5b4b` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:43.758780 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-f7e1c8cf-24d5-4235-9c72-3bf7392b3c38 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '838adbfb1b7b4b35bfd49dde18d5d9e4'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:43.759724 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0404efc-3fe1-4c24-b2b4-e6d1a159f4ce admin admin] RBAC: Authorizing `identity:delete_user(user_id=c38f9d253f024686ba765c6a7fec5b4b)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:43.762156 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0404efc-3fe1-4c24-b2b4-e6d1a159f4ce admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:43.769326 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-f7e1c8cf-24d5-4235-9c72-3bf7392b3c38 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '838adbfb1b7b4b35bfd49dde18d5d9e4'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:43.779670 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-f7e1c8cf-24d5-4235-9c72-3bf7392b3c38 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '838adbfb1b7b4b35bfd49dde18d5d9e4'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:43.781189 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-f7e1c8cf-24d5-4235-9c72-3bf7392b3c38 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '838adbfb1b7b4b35bfd49dde18d5d9e4'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:43.781713 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-f7e1c8cf-24d5-4235-9c72-3bf7392b3c38 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '838adbfb1b7b4b35bfd49dde18d5d9e4'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:43.812916 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 962/1954] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Jul 27 10:48:43 2022] DELETE /identity/v3/users/838adbfb1b7b4b35bfd49dde18d5d9e4 => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:48:43.814139 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e1c8cf-24d5-4235-9c72-3bf7392b3c38 admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:43.816259 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-a0404efc-3fe1-4c24-b2b4-e6d1a159f4ce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c38f9d253f024686ba765c6a7fec5b4b'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:43.816374 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-f7e1c8cf-24d5-4235-9c72-3bf7392b3c38 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:43.824542 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-a0404efc-3fe1-4c24-b2b4-e6d1a159f4ce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c38f9d253f024686ba765c6a7fec5b4b'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:43.834278 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-a0404efc-3fe1-4c24-b2b4-e6d1a159f4ce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c38f9d253f024686ba765c6a7fec5b4b'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:43.835318 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-a0404efc-3fe1-4c24-b2b4-e6d1a159f4ce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c38f9d253f024686ba765c6a7fec5b4b'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:43.836151 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-a0404efc-3fe1-4c24-b2b4-e6d1a159f4ce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c38f9d253f024686ba765c6a7fec5b4b'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:43.859425 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 993/1955] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Jul 27 10:48:43 2022] DELETE /identity/v3/users/c38f9d253f024686ba765c6a7fec5b4b => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:48:43.859627 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0404efc-3fe1-4c24-b2b4-e6d1a159f4ce admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:43.861001 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-a0404efc-3fe1-4c24-b2b4-e6d1a159f4ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:43.868987 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:48:43.920565 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e1c8cf-24d5-4235-9c72-3bf7392b3c38 admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:43.954283 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0404efc-3fe1-4c24-b2b4-e6d1a159f4ce admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:43.958540 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2875764-37bf-402e-acfb-8c45420ce980 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:43.959452 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2875764-37bf-402e-acfb-8c45420ce980 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:43.959829 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2875764-37bf-402e-acfb-8c45420ce980 admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:43.960847 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2875764-37bf-402e-acfb-8c45420ce980 admin admin] PATH_INFO: `/v3/projects/37b26801bd514df6ae0d722db67cffeb` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:43.962541 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2875764-37bf-402e-acfb-8c45420ce980 admin admin] RBAC: Authorizing `identity:delete_project(project_id=37b26801bd514df6ae0d722db67cffeb)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:43.965406 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2875764-37bf-402e-acfb-8c45420ce980 admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:44.026940 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d674dd3-5e56-4579-ad54-90b19dd0f344 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:44.028603 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d674dd3-5e56-4579-ad54-90b19dd0f344 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:44.029268 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d674dd3-5e56-4579-ad54-90b19dd0f344 admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:44.029600 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d674dd3-5e56-4579-ad54-90b19dd0f344 admin admin] PATH_INFO: `/v3/users/d51214bfb27e4f0696f0c66411f35fe9` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:44.031207 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d674dd3-5e56-4579-ad54-90b19dd0f344 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d51214bfb27e4f0696f0c66411f35fe9)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:44.032297 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-d2875764-37bf-402e-acfb-8c45420ce980 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '37b26801bd514df6ae0d722db67cffeb', 'request_id': 'req-d2875764-37bf-402e-acfb-8c45420ce980'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:44.033419 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d674dd3-5e56-4579-ad54-90b19dd0f344 admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:44.039729 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-d2875764-37bf-402e-acfb-8c45420ce980 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '37b26801bd514df6ae0d722db67cffeb', 'request_id': 'req-d2875764-37bf-402e-acfb-8c45420ce980'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:44.047142 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-d2875764-37bf-402e-acfb-8c45420ce980 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '37b26801bd514df6ae0d722db67cffeb', 'request_id': 'req-d2875764-37bf-402e-acfb-8c45420ce980'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:44.059175 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-d2875764-37bf-402e-acfb-8c45420ce980 admin admin] The token cache is being invalidate because project 37b26801bd514df6ae0d722db67cffeb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86827) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 10:48:44.059483 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-d2875764-37bf-402e-acfb-8c45420ce980 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:44.060134 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-d2875764-37bf-402e-acfb-8c45420ce980 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:44.061951 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 994/1956] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Jul 27 10:48:43 2022] DELETE /identity/v3/projects/37b26801bd514df6ae0d722db67cffeb => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:48:44.062759 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2875764-37bf-402e-acfb-8c45420ce980 admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:44.064441 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-d2875764-37bf-402e-acfb-8c45420ce980 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:44.088641 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:DELETE=15,keystone:SELECT=84,keystone:INSERT=1 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:48:44.093695 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-8d674dd3-5e56-4579-ad54-90b19dd0f344 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd51214bfb27e4f0696f0c66411f35fe9'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:44.102637 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-8d674dd3-5e56-4579-ad54-90b19dd0f344 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd51214bfb27e4f0696f0c66411f35fe9'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:44.111166 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-8d674dd3-5e56-4579-ad54-90b19dd0f344 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd51214bfb27e4f0696f0c66411f35fe9'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:44.111861 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-8d674dd3-5e56-4579-ad54-90b19dd0f344 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd51214bfb27e4f0696f0c66411f35fe9'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:44.112534 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-8d674dd3-5e56-4579-ad54-90b19dd0f344 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd51214bfb27e4f0696f0c66411f35fe9'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:44.133797 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 963/1957] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Jul 27 10:48:43 2022] DELETE /identity/v3/users/d51214bfb27e4f0696f0c66411f35fe9 => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:48:44.135170 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6473bb44-ad2f-4e27-8afa-21d3fc9d7174 None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:44.135459 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6473bb44-ad2f-4e27-8afa-21d3fc9d7174 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:44.135731 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6473bb44-ad2f-4e27-8afa-21d3fc9d7174 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:44.138653 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2875764-37bf-402e-acfb-8c45420ce980 admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:44.157508 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-6473bb44-ad2f-4e27-8afa-21d3fc9d7174 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:48:44.158750 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-6473bb44-ad2f-4e27-8afa-21d3fc9d7174 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:44.205743 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a391ee39-9778-4cd7-b9ce-43555a2f620c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7922d0a1a7c4e808d0b2a80dca0da40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:44.206381 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a391ee39-9778-4cd7-b9ce-43555a2f620c service neutron] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:44.206942 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a391ee39-9778-4cd7-b9ce-43555a2f620c service neutron] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:44.207265 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a391ee39-9778-4cd7-b9ce-43555a2f620c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:44.209189 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-a391ee39-9778-4cd7-b9ce-43555a2f620c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:44.219724 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:48:44.268213 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 964/1958] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:48:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:48:44.272771 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6473bb44-ad2f-4e27-8afa-21d3fc9d7174 None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:44.274376 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-6473bb44-ad2f-4e27-8afa-21d3fc9d7174 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:44.290499 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6473bb44-ad2f-4e27-8afa-21d3fc9d7174 None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:44.297365 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a3e6c7-7df0-4a33-9ba4-537a891ea653 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:44.298400 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a3e6c7-7df0-4a33-9ba4-537a891ea653 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:44.298920 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a3e6c7-7df0-4a33-9ba4-537a891ea653 admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:44.299417 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a3e6c7-7df0-4a33-9ba4-537a891ea653 admin admin] PATH_INFO: `/v3/users/ef39ebeec06a411dbd6f6176e71d2eb3` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:44.300856 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a3e6c7-7df0-4a33-9ba4-537a891ea653 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ef39ebeec06a411dbd6f6176e71d2eb3)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:44.303881 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a3e6c7-7df0-4a33-9ba4-537a891ea653 admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:44.353669 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a391ee39-9778-4cd7-b9ce-43555a2f620c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:44.358545 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a391ee39-9778-4cd7-b9ce-43555a2f620c service neutron] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:44.377001 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 995/1959] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:48:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 10:48:44.377824 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a391ee39-9778-4cd7-b9ce-43555a2f620c service neutron] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:44.380301 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-a391ee39-9778-4cd7-b9ce-43555a2f620c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:44.396209 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a391ee39-9778-4cd7-b9ce-43555a2f620c service neutron] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:44.401721 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41a5e09-498c-4fcb-b0d2-51249d472c14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:44.401968 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41a5e09-498c-4fcb-b0d2-51249d472c14 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:44.402296 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41a5e09-498c-4fcb-b0d2-51249d472c14 admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:44.402579 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41a5e09-498c-4fcb-b0d2-51249d472c14 admin admin] PATH_INFO: `/v3/projects/13d6ee0c8c9740c3a581b97b8f370525` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:44.403830 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41a5e09-498c-4fcb-b0d2-51249d472c14 admin admin] RBAC: Authorizing `identity:delete_project(project_id=13d6ee0c8c9740c3a581b97b8f370525)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:44.404921 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-71a3e6c7-7df0-4a33-9ba4-537a891ea653 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ef39ebeec06a411dbd6f6176e71d2eb3'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:44.406508 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41a5e09-498c-4fcb-b0d2-51249d472c14 admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:44.416195 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-71a3e6c7-7df0-4a33-9ba4-537a891ea653 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ef39ebeec06a411dbd6f6176e71d2eb3'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:44.429947 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-71a3e6c7-7df0-4a33-9ba4-537a891ea653 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ef39ebeec06a411dbd6f6176e71d2eb3'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:44.431271 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-71a3e6c7-7df0-4a33-9ba4-537a891ea653 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ef39ebeec06a411dbd6f6176e71d2eb3'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:44.432297 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-71a3e6c7-7df0-4a33-9ba4-537a891ea653 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ef39ebeec06a411dbd6f6176e71d2eb3'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:44.483809 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 965/1960] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Jul 27 10:48:44 2022] DELETE /identity/v3/users/ef39ebeec06a411dbd6f6176e71d2eb3 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:48:44.486132 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a3e6c7-7df0-4a33-9ba4-537a891ea653 admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:44.490528 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-71a3e6c7-7df0-4a33-9ba4-537a891ea653 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:44.496711 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-f41a5e09-498c-4fcb-b0d2-51249d472c14 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '13d6ee0c8c9740c3a581b97b8f370525', 'request_id': 'req-f41a5e09-498c-4fcb-b0d2-51249d472c14'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:44.512576 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-f41a5e09-498c-4fcb-b0d2-51249d472c14 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '13d6ee0c8c9740c3a581b97b8f370525', 'request_id': 'req-f41a5e09-498c-4fcb-b0d2-51249d472c14'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:44.529848 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-f41a5e09-498c-4fcb-b0d2-51249d472c14 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '13d6ee0c8c9740c3a581b97b8f370525', 'request_id': 'req-f41a5e09-498c-4fcb-b0d2-51249d472c14'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:44.555642 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-f41a5e09-498c-4fcb-b0d2-51249d472c14 admin admin] The token cache is being invalidate because project 13d6ee0c8c9740c3a581b97b8f370525 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86827) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 10:48:44.555742 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-f41a5e09-498c-4fcb-b0d2-51249d472c14 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:44.557359 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-f41a5e09-498c-4fcb-b0d2-51249d472c14 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:44.560461 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 996/1961] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Jul 27 10:48:44 2022] DELETE /identity/v3/projects/13d6ee0c8c9740c3a581b97b8f370525 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:48:44.567726 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06866df-1560-4704-9890-80f27c290c5f None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:44.568033 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06866df-1560-4704-9890-80f27c290c5f None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:44.568295 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06866df-1560-4704-9890-80f27c290c5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:44.579680 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:48:44.612606 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a3e6c7-7df0-4a33-9ba4-537a891ea653 admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:44.614643 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-a06866df-1560-4704-9890-80f27c290c5f None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:48:44.615095 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-a06866df-1560-4704-9890-80f27c290c5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:44.617798 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd1a826-6632-4c16-baac-ac2502601254 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7922d0a1a7c4e808d0b2a80dca0da40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:44.618449 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd1a826-6632-4c16-baac-ac2502601254 service neutron] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:44.618802 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd1a826-6632-4c16-baac-ac2502601254 service neutron] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:44.619080 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd1a826-6632-4c16-baac-ac2502601254 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:44.625425 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-1dd1a826-6632-4c16-baac-ac2502601254 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:44.739257 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd1a826-6632-4c16-baac-ac2502601254 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:44.743201 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd1a826-6632-4c16-baac-ac2502601254 service neutron] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:44.778286 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 966/1962] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:48:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 10:48:44.778892 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 997/1963] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:48:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:48:44.785234 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c519a9-11fa-4f95-9ca9-0be44b634387 None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:44.785778 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c519a9-11fa-4f95-9ca9-0be44b634387 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:44.786151 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c519a9-11fa-4f95-9ca9-0be44b634387 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:44.810449 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-c9c519a9-11fa-4f95-9ca9-0be44b634387 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:48:44.812198 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-c9c519a9-11fa-4f95-9ca9-0be44b634387 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:44.819621 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06866df-1560-4704-9890-80f27c290c5f None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:44.821159 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-a06866df-1560-4704-9890-80f27c290c5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:44.835977 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 967/1964] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:48:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 10:48:44.840281 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c519a9-11fa-4f95-9ca9-0be44b634387 None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:44.842026 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06866df-1560-4704-9890-80f27c290c5f None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:44.852626 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff63af07-8673-4265-99b8-d7aa646e25ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:44.855892 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff63af07-8673-4265-99b8-d7aa646e25ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:44.856275 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff63af07-8673-4265-99b8-d7aa646e25ee admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:44.858408 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff63af07-8673-4265-99b8-d7aa646e25ee admin admin] PATH_INFO: `/v3/projects/dd36bbdd9cde482b9c79b928ddc8b0e7` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:44.861430 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff63af07-8673-4265-99b8-d7aa646e25ee admin admin] RBAC: Authorizing `identity:delete_project(project_id=dd36bbdd9cde482b9c79b928ddc8b0e7)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:44.864002 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c519a9-11fa-4f95-9ca9-0be44b634387 None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:44.870107 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff63af07-8673-4265-99b8-d7aa646e25ee admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:44.878312 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bde65cf-ce59-46be-ad0a-c10478173915 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:44.881933 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bde65cf-ce59-46be-ad0a-c10478173915 admin admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:44.882370 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bde65cf-ce59-46be-ad0a-c10478173915 admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:44.882754 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bde65cf-ce59-46be-ad0a-c10478173915 admin admin] PATH_INFO: `/v3/domains` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:44.883577 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bde65cf-ce59-46be-ad0a-c10478173915 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86828) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:48:44.883961 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bde65cf-ce59-46be-ad0a-c10478173915 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:44.889550 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bde65cf-ce59-46be-ad0a-c10478173915 admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:44.921207 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 968/1965] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Jul 27 10:48:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:48:44.925986 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bde65cf-ce59-46be-ad0a-c10478173915 admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:44.934498 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:48:44.961496 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bde65cf-ce59-46be-ad0a-c10478173915 admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:44.975603 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ad3e8f-e4a2-4ba6-98a7-32ebeecb03b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:44.976731 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ad3e8f-e4a2-4ba6-98a7-32ebeecb03b5 admin admin] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:44.977786 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ad3e8f-e4a2-4ba6-98a7-32ebeecb03b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:44.978745 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ad3e8f-e4a2-4ba6-98a7-32ebeecb03b5 admin admin] PATH_INFO: `/v3/projects` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:44.979443 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ad3e8f-e4a2-4ba6-98a7-32ebeecb03b5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:44.984454 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ad3e8f-e4a2-4ba6-98a7-32ebeecb03b5 admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:44.998225 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-ff63af07-8673-4265-99b8-d7aa646e25ee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dd36bbdd9cde482b9c79b928ddc8b0e7', 'request_id': 'req-ff63af07-8673-4265-99b8-d7aa646e25ee'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:45.012984 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-ff63af07-8673-4265-99b8-d7aa646e25ee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dd36bbdd9cde482b9c79b928ddc8b0e7', 'request_id': 'req-ff63af07-8673-4265-99b8-d7aa646e25ee'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:45.029609 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-ff63af07-8673-4265-99b8-d7aa646e25ee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dd36bbdd9cde482b9c79b928ddc8b0e7', 'request_id': 'req-ff63af07-8673-4265-99b8-d7aa646e25ee'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:45.061515 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-ff63af07-8673-4265-99b8-d7aa646e25ee admin admin] The token cache is being invalidate because project dd36bbdd9cde482b9c79b928ddc8b0e7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86827) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 10:48:45.061907 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-ff63af07-8673-4265-99b8-d7aa646e25ee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:45.063318 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-ff63af07-8673-4265-99b8-d7aa646e25ee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:45.070764 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 998/1966] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Jul 27 10:48:44 2022] DELETE /identity/v3/projects/dd36bbdd9cde482b9c79b928ddc8b0e7 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:48:45.072670 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 969/1967] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Jul 27 10:48:44 2022] POST /identity/v3/projects => generated 375 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 10:48:45.078252 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5229d65-6d0e-4a68-bc02-7ee69bb43c8e None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:45.078454 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5229d65-6d0e-4a68-bc02-7ee69bb43c8e None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:45.078816 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5229d65-6d0e-4a68-bc02-7ee69bb43c8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:45.089150 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ad3e8f-e4a2-4ba6-98a7-32ebeecb03b5 admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:45.099957 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-d2ad3e8f-e4a2-4ba6-98a7-32ebeecb03b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:45.117349 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-d5229d65-6d0e-4a68-bc02-7ee69bb43c8e None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:48:45.120372 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-d5229d65-6d0e-4a68-bc02-7ee69bb43c8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:45.216514 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 999/1968] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:48:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:48:45.224325 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ad3e8f-e4a2-4ba6-98a7-32ebeecb03b5 admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:45.224818 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b559ef1a-fead-46f4-8921-6a446964b5a3 None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:45.225207 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b559ef1a-fead-46f4-8921-6a446964b5a3 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:45.225491 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b559ef1a-fead-46f4-8921-6a446964b5a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:45.233638 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665abca7-f124-4b94-8012-0e568353db20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:45.234791 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665abca7-f124-4b94-8012-0e568353db20 admin admin] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:45.235659 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665abca7-f124-4b94-8012-0e568353db20 admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:45.236133 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665abca7-f124-4b94-8012-0e568353db20 admin admin] PATH_INFO: `/v3/users` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:45.237821 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665abca7-f124-4b94-8012-0e568353db20 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:45.241789 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665abca7-f124-4b94-8012-0e568353db20 admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:45.246798 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-b559ef1a-fead-46f4-8921-6a446964b5a3 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:48:45.248857 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-b559ef1a-fead-46f4-8921-6a446964b5a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:45.263785 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1000/1969] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:48:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:48:45.265301 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b559ef1a-fead-46f4-8921-6a446964b5a3 None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:45.281473 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b559ef1a-fead-46f4-8921-6a446964b5a3 None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:45.286590 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 970/1970] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Jul 27 10:48:45 2022] POST /identity/v3/users => generated 288 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 10:48:45.289564 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df463236-bc3b-422b-a15b-850a0f67ad74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:45.290372 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df463236-bc3b-422b-a15b-850a0f67ad74 admin admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:45.290741 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df463236-bc3b-422b-a15b-850a0f67ad74 admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:45.291047 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df463236-bc3b-422b-a15b-850a0f67ad74 admin admin] PATH_INFO: `/v3/domains` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:45.292096 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df463236-bc3b-422b-a15b-850a0f67ad74 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86827) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:48:45.292096 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df463236-bc3b-422b-a15b-850a0f67ad74 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:45.292967 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665abca7-f124-4b94-8012-0e568353db20 admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:45.293934 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df463236-bc3b-422b-a15b-850a0f67ad74 admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:45.314893 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1001/1971] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Jul 27 10:48:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:48:45.317448 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df463236-bc3b-422b-a15b-850a0f67ad74 admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:45.332063 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665abca7-f124-4b94-8012-0e568353db20 admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:45.365405 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33324c1-8693-4b6f-b294-9b128ab4d95f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:45.367110 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33324c1-8693-4b6f-b294-9b128ab4d95f admin admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:45.367557 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33324c1-8693-4b6f-b294-9b128ab4d95f admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:45.368181 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33324c1-8693-4b6f-b294-9b128ab4d95f admin admin] PATH_INFO: `/v3/roles` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:45.369293 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33324c1-8693-4b6f-b294-9b128ab4d95f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:45.372503 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33324c1-8693-4b6f-b294-9b128ab4d95f admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:45.374808 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df463236-bc3b-422b-a15b-850a0f67ad74 admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:45.387081 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 971/1972] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 10:48:45 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 10:48:45.389383 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33324c1-8693-4b6f-b294-9b128ab4d95f admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:45.391764 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-b33324c1-8693-4b6f-b294-9b128ab4d95f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:45.396083 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753d621d-39e5-4717-aeb1-67048657bb41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:45.396722 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753d621d-39e5-4717-aeb1-67048657bb41 admin admin] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:45.397003 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753d621d-39e5-4717-aeb1-67048657bb41 admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:45.397276 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753d621d-39e5-4717-aeb1-67048657bb41 admin admin] PATH_INFO: `/v3/projects` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:45.398282 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753d621d-39e5-4717-aeb1-67048657bb41 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:45.400226 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753d621d-39e5-4717-aeb1-67048657bb41 admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:45.428454 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33324c1-8693-4b6f-b294-9b128ab4d95f admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:45.435229 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab7b868-6d2c-4ad2-bc8d-7293e3303e62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:45.435915 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab7b868-6d2c-4ad2-bc8d-7293e3303e62 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:45.436203 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab7b868-6d2c-4ad2-bc8d-7293e3303e62 admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:45.436481 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab7b868-6d2c-4ad2-bc8d-7293e3303e62 admin admin] PATH_INFO: `/v3/projects/1ea4ccfa264a47499fa5bdcee98c046e` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:45.437749 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab7b868-6d2c-4ad2-bc8d-7293e3303e62 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1ea4ccfa264a47499fa5bdcee98c046e)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:45.440574 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab7b868-6d2c-4ad2-bc8d-7293e3303e62 admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:45.460918 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1002/1973] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Jul 27 10:48:45 2022] POST /identity/v3/projects => generated 383 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 10:48:45.461914 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753d621d-39e5-4717-aeb1-67048657bb41 admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:45.465883 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-753d621d-39e5-4717-aeb1-67048657bb41 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:45.474287 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:48:45.515873 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-7ab7b868-6d2c-4ad2-bc8d-7293e3303e62 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1ea4ccfa264a47499fa5bdcee98c046e', 'request_id': 'req-7ab7b868-6d2c-4ad2-bc8d-7293e3303e62'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:45.525442 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-7ab7b868-6d2c-4ad2-bc8d-7293e3303e62 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1ea4ccfa264a47499fa5bdcee98c046e', 'request_id': 'req-7ab7b868-6d2c-4ad2-bc8d-7293e3303e62'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:45.551852 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-7ab7b868-6d2c-4ad2-bc8d-7293e3303e62 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1ea4ccfa264a47499fa5bdcee98c046e', 'request_id': 'req-7ab7b868-6d2c-4ad2-bc8d-7293e3303e62'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:45.552056 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753d621d-39e5-4717-aeb1-67048657bb41 admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:45.562192 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:48:45.591665 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-7ab7b868-6d2c-4ad2-bc8d-7293e3303e62 admin admin] The token cache is being invalidate because project 1ea4ccfa264a47499fa5bdcee98c046e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86828) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 10:48:45.602583 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-7ab7b868-6d2c-4ad2-bc8d-7293e3303e62 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:45.607681 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-7ab7b868-6d2c-4ad2-bc8d-7293e3303e62 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:45.612667 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 972/1974] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Jul 27 10:48:45 2022] DELETE /identity/v3/projects/1ea4ccfa264a47499fa5bdcee98c046e => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:48:45.612667 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab7b868-6d2c-4ad2-bc8d-7293e3303e62 admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:45.613962 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-7ab7b868-6d2c-4ad2-bc8d-7293e3303e62 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:45.624477 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505f08dd-7d57-46ef-beee-75593eff6a0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:45.624882 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505f08dd-7d57-46ef-beee-75593eff6a0d admin admin] REQUEST_METHOD: `DELETE` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:45.625171 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505f08dd-7d57-46ef-beee-75593eff6a0d admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:45.625453 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505f08dd-7d57-46ef-beee-75593eff6a0d admin admin] PATH_INFO: `/v3/projects/5254304966a9423e867fef253079bb47` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:45.627063 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505f08dd-7d57-46ef-beee-75593eff6a0d admin admin] RBAC: Authorizing `identity:delete_project(project_id=5254304966a9423e867fef253079bb47)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:45.628996 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505f08dd-7d57-46ef-beee-75593eff6a0d admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:45.706308 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-505f08dd-7d57-46ef-beee-75593eff6a0d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5254304966a9423e867fef253079bb47', 'request_id': 'req-505f08dd-7d57-46ef-beee-75593eff6a0d'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:45.715753 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-505f08dd-7d57-46ef-beee-75593eff6a0d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5254304966a9423e867fef253079bb47', 'request_id': 'req-505f08dd-7d57-46ef-beee-75593eff6a0d'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:45.725774 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab7b868-6d2c-4ad2-bc8d-7293e3303e62 admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:45.728767 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-505f08dd-7d57-46ef-beee-75593eff6a0d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5254304966a9423e867fef253079bb47', 'request_id': 'req-505f08dd-7d57-46ef-beee-75593eff6a0d'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:45.741471 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-505f08dd-7d57-46ef-beee-75593eff6a0d admin admin] The token cache is being invalidate because project 5254304966a9423e867fef253079bb47 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86827) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 10:48:45.741782 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-505f08dd-7d57-46ef-beee-75593eff6a0d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:45.742436 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-505f08dd-7d57-46ef-beee-75593eff6a0d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:45.744072 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1003/1975] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Jul 27 10:48:45 2022] DELETE /identity/v3/projects/5254304966a9423e867fef253079bb47 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:48:45.744815 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505f08dd-7d57-46ef-beee-75593eff6a0d admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:45.746876 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-505f08dd-7d57-46ef-beee-75593eff6a0d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:45.803936 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10063bc2-c719-4395-bdb7-ee22b76af6b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:45.804805 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10063bc2-c719-4395-bdb7-ee22b76af6b1 admin admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:45.805177 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10063bc2-c719-4395-bdb7-ee22b76af6b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:45.805533 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10063bc2-c719-4395-bdb7-ee22b76af6b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:45.806302 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10063bc2-c719-4395-bdb7-ee22b76af6b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:45.808027 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10063bc2-c719-4395-bdb7-ee22b76af6b1 admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:45.823788 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 973/1976] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 10:48:45 2022] GET /identity/v3/roles => generated 1420 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 10:48:45.823788 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10063bc2-c719-4395-bdb7-ee22b76af6b1 admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:45.825308 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-10063bc2-c719-4395-bdb7-ee22b76af6b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:45.838475 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10063bc2-c719-4395-bdb7-ee22b76af6b1 admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:45.843491 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6f437e-fa36-4188-a0ca-7763eaec727e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:45.844071 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6f437e-fa36-4188-a0ca-7763eaec727e admin admin] REQUEST_METHOD: `DELETE` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:45.844346 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6f437e-fa36-4188-a0ca-7763eaec727e admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:45.844630 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6f437e-fa36-4188-a0ca-7763eaec727e admin admin] PATH_INFO: `/v3/users/b3f90d3b43b14140ade93463d368c963` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:45.845895 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6f437e-fa36-4188-a0ca-7763eaec727e admin admin] RBAC: Authorizing `identity:delete_user(user_id=b3f90d3b43b14140ade93463d368c963)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:45.847235 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505f08dd-7d57-46ef-beee-75593eff6a0d admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:45.848072 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6f437e-fa36-4188-a0ca-7763eaec727e admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:45.852642 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1f38ae-33e5-4f45-907b-827c101ccf28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:45.853352 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1f38ae-33e5-4f45-907b-827c101ccf28 admin admin] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:45.853707 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1f38ae-33e5-4f45-907b-827c101ccf28 admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:45.854640 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1f38ae-33e5-4f45-907b-827c101ccf28 admin admin] PATH_INFO: `/v3/users` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:45.854785 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1f38ae-33e5-4f45-907b-827c101ccf28 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:45.856765 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1f38ae-33e5-4f45-907b-827c101ccf28 admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:45.884262 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1004/1977] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Jul 27 10:48:45 2022] POST /identity/v3/users => generated 292 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 10:48:45.885144 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1f38ae-33e5-4f45-907b-827c101ccf28 admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:45.887407 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-bd1f38ae-33e5-4f45-907b-827c101ccf28 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:45.899366 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1f38ae-33e5-4f45-907b-827c101ccf28 admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:45.900829 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:48:45.904343 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aae3798-b443-4ec4-845e-f014d918d96e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:45.904891 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aae3798-b443-4ec4-845e-f014d918d96e admin admin] REQUEST_METHOD: `DELETE` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:45.905257 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aae3798-b443-4ec4-845e-f014d918d96e admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:45.905514 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aae3798-b443-4ec4-845e-f014d918d96e admin admin] PATH_INFO: `/v3/projects/cb55b64bad614375827066b02150d6bb` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:45.906659 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aae3798-b443-4ec4-845e-f014d918d96e admin admin] RBAC: Authorizing `identity:delete_project(project_id=cb55b64bad614375827066b02150d6bb)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:45.907404 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-0d6f437e-fa36-4188-a0ca-7763eaec727e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b3f90d3b43b14140ade93463d368c963'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:45.908624 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aae3798-b443-4ec4-845e-f014d918d96e admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:45.915749 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-0d6f437e-fa36-4188-a0ca-7763eaec727e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b3f90d3b43b14140ade93463d368c963'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:45.923243 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-0d6f437e-fa36-4188-a0ca-7763eaec727e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b3f90d3b43b14140ade93463d368c963'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:45.924018 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-0d6f437e-fa36-4188-a0ca-7763eaec727e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b3f90d3b43b14140ade93463d368c963'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:45.924674 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-0d6f437e-fa36-4188-a0ca-7763eaec727e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b3f90d3b43b14140ade93463d368c963'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:45.944363 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:48:45.957968 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 974/1978] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Jul 27 10:48:45 2022] DELETE /identity/v3/users/b3f90d3b43b14140ade93463d368c963 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:48:45.962269 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e6f1a2-7cb0-40ed-aaf6-23affa9d991b None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:45.962564 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e6f1a2-7cb0-40ed-aaf6-23affa9d991b None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:45.962865 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e6f1a2-7cb0-40ed-aaf6-23affa9d991b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:45.976784 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-1aae3798-b443-4ec4-845e-f014d918d96e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cb55b64bad614375827066b02150d6bb', 'request_id': 'req-1aae3798-b443-4ec4-845e-f014d918d96e'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:45.986587 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-1aae3798-b443-4ec4-845e-f014d918d96e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cb55b64bad614375827066b02150d6bb', 'request_id': 'req-1aae3798-b443-4ec4-845e-f014d918d96e'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:45.987986 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-18e6f1a2-7cb0-40ed-aaf6-23affa9d991b None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:48:45.989421 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-18e6f1a2-7cb0-40ed-aaf6-23affa9d991b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:45.997059 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-1aae3798-b443-4ec4-845e-f014d918d96e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cb55b64bad614375827066b02150d6bb', 'request_id': 'req-1aae3798-b443-4ec4-845e-f014d918d96e'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:46.010320 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-1aae3798-b443-4ec4-845e-f014d918d96e admin admin] The token cache is being invalidate because project cb55b64bad614375827066b02150d6bb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86827) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 10:48:46.010973 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-1aae3798-b443-4ec4-845e-f014d918d96e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:46.011785 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-1aae3798-b443-4ec4-845e-f014d918d96e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:46.016294 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1005/1979] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Jul 27 10:48:45 2022] DELETE /identity/v3/projects/cb55b64bad614375827066b02150d6bb => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:48:46.016294 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aae3798-b443-4ec4-845e-f014d918d96e admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:46.017006 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-1aae3798-b443-4ec4-845e-f014d918d96e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:46.074542 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 975/1980] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:48:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:48:46.075774 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e6f1a2-7cb0-40ed-aaf6-23affa9d991b None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:46.077437 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-18e6f1a2-7cb0-40ed-aaf6-23affa9d991b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:46.092852 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e6f1a2-7cb0-40ed-aaf6-23affa9d991b None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:46.093080 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aae3798-b443-4ec4-845e-f014d918d96e admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:46.097941 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b68abb1-3ecf-42ad-b836-4d5d62edb368 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:46.098575 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b68abb1-3ecf-42ad-b836-4d5d62edb368 admin admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:46.098937 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b68abb1-3ecf-42ad-b836-4d5d62edb368 admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:46.099210 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b68abb1-3ecf-42ad-b836-4d5d62edb368 admin admin] PATH_INFO: `/v3/roles` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:46.099321 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43235968-e61d-4c64-a97d-b7c4952d754d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:46.099909 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b68abb1-3ecf-42ad-b836-4d5d62edb368 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:46.100021 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43235968-e61d-4c64-a97d-b7c4952d754d admin admin] REQUEST_METHOD: `PUT` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:46.100277 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43235968-e61d-4c64-a97d-b7c4952d754d admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:46.100922 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43235968-e61d-4c64-a97d-b7c4952d754d admin admin] PATH_INFO: `/v3/projects/c3719baac0324707b90c1823647f047e/users/2f81981f4c184f6cb9bef624d7096b35/roles/4c9fc0b3baed441da2126be41cc6b04a` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:46.101507 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b68abb1-3ecf-42ad-b836-4d5d62edb368 admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:46.118788 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 976/1981] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 10:48:46 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 10:48:46.120028 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b68abb1-3ecf-42ad-b836-4d5d62edb368 admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:46.122678 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43235968-e61d-4c64-a97d-b7c4952d754d admin admin] RBAC: Authorizing `identity:create_grant(project_id=c3719baac0324707b90c1823647f047e, user_id=2f81981f4c184f6cb9bef624d7096b35, role_id=4c9fc0b3baed441da2126be41cc6b04a)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:46.125619 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43235968-e61d-4c64-a97d-b7c4952d754d admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:46.127240 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-8b68abb1-3ecf-42ad-b836-4d5d62edb368 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:46.158165 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1006/1982] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 10:48:46 2022] PUT /identity/v3/projects/c3719baac0324707b90c1823647f047e/users/2f81981f4c184f6cb9bef624d7096b35/roles/4c9fc0b3baed441da2126be41cc6b04a => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:48:46.158165 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc681107-6e33-4a6b-9e78-8b5694a1d1ae None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:46.158165 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc681107-6e33-4a6b-9e78-8b5694a1d1ae None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:46.158165 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc681107-6e33-4a6b-9e78-8b5694a1d1ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:46.158591 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b68abb1-3ecf-42ad-b836-4d5d62edb368 admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:46.191166 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-dc681107-6e33-4a6b-9e78-8b5694a1d1ae None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:48:46.192605 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-dc681107-6e33-4a6b-9e78-8b5694a1d1ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:46.229729 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995442d0-990a-46da-a600-4b99e6b24e02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:46.231335 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995442d0-990a-46da-a600-4b99e6b24e02 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:46.231335 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995442d0-990a-46da-a600-4b99e6b24e02 admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:46.231536 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995442d0-990a-46da-a600-4b99e6b24e02 admin admin] PATH_INFO: `/v3/users/bfcc4bf290e442de848d0d07a4119e01` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:46.232959 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995442d0-990a-46da-a600-4b99e6b24e02 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bfcc4bf290e442de848d0d07a4119e01)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:46.234952 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995442d0-990a-46da-a600-4b99e6b24e02 admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:46.263030 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1007/1983] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:48:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:48:46.264174 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc681107-6e33-4a6b-9e78-8b5694a1d1ae None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:46.276458 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc681107-6e33-4a6b-9e78-8b5694a1d1ae None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:46.279626 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3976e2d1-e76e-41e1-8aba-cd9c07e3564c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:46.280258 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3976e2d1-e76e-41e1-8aba-cd9c07e3564c admin admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:46.280540 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3976e2d1-e76e-41e1-8aba-cd9c07e3564c admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:46.280817 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3976e2d1-e76e-41e1-8aba-cd9c07e3564c admin admin] PATH_INFO: `/v3/roles` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:46.282132 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3976e2d1-e76e-41e1-8aba-cd9c07e3564c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:46.283838 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3976e2d1-e76e-41e1-8aba-cd9c07e3564c admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:46.298141 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1008/1984] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 10:48:46 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 10:48:46.298141 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3611f4b-e398-4f61-b022-b4c79ff72fe3 None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:46.298141 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3611f4b-e398-4f61-b022-b4c79ff72fe3 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:46.298141 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3611f4b-e398-4f61-b022-b4c79ff72fe3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:46.301066 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-995442d0-990a-46da-a600-4b99e6b24e02 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bfcc4bf290e442de848d0d07a4119e01'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:46.303428 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:48:46.315501 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-995442d0-990a-46da-a600-4b99e6b24e02 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bfcc4bf290e442de848d0d07a4119e01'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:46.323492 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-995442d0-990a-46da-a600-4b99e6b24e02 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bfcc4bf290e442de848d0d07a4119e01'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:46.324341 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-995442d0-990a-46da-a600-4b99e6b24e02 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bfcc4bf290e442de848d0d07a4119e01'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:46.325230 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-995442d0-990a-46da-a600-4b99e6b24e02 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bfcc4bf290e442de848d0d07a4119e01'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:46.327140 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-e3611f4b-e398-4f61-b022-b4c79ff72fe3 None None] MFA Rules not processed for user `2f81981f4c184f6cb9bef624d7096b35`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:48:46.328359 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-e3611f4b-e398-4f61-b022-b4c79ff72fe3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:46.345613 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 977/1985] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Jul 27 10:48:46 2022] DELETE /identity/v3/users/bfcc4bf290e442de848d0d07a4119e01 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:48:46.346510 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:48:46.351542 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995442d0-990a-46da-a600-4b99e6b24e02 admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:46.352986 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-995442d0-990a-46da-a600-4b99e6b24e02 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:46.438474 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1009/1986] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:48:46 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:48:46.442084 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3611f4b-e398-4f61-b022-b4c79ff72fe3 None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:46.442084 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-e3611f4b-e398-4f61-b022-b4c79ff72fe3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:46.457354 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3611f4b-e398-4f61-b022-b4c79ff72fe3 None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:46.462426 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2329cba2-3a53-460e-9de8-2964215e3751 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:46.463281 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2329cba2-3a53-460e-9de8-2964215e3751 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:46.463625 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2329cba2-3a53-460e-9de8-2964215e3751 admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:46.464067 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2329cba2-3a53-460e-9de8-2964215e3751 admin admin] PATH_INFO: `/v3/projects/50958e3a173847db998537e22c8f3efb` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:46.466259 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2329cba2-3a53-460e-9de8-2964215e3751 admin admin] RBAC: Authorizing `identity:delete_project(project_id=50958e3a173847db998537e22c8f3efb)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:46.468671 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2329cba2-3a53-460e-9de8-2964215e3751 admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:46.476266 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995442d0-990a-46da-a600-4b99e6b24e02 admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:46.482153 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b82d51-989f-4c86-85c8-09f0a3a68540 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:46.482828 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b82d51-989f-4c86-85c8-09f0a3a68540 admin admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:46.483116 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b82d51-989f-4c86-85c8-09f0a3a68540 admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:46.483886 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b82d51-989f-4c86-85c8-09f0a3a68540 admin admin] PATH_INFO: `/v3/domains` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:46.484607 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b82d51-989f-4c86-85c8-09f0a3a68540 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86828) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:48:46.484917 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b82d51-989f-4c86-85c8-09f0a3a68540 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:46.487105 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b82d51-989f-4c86-85c8-09f0a3a68540 admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:46.504157 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 978/1987] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Jul 27 10:48:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:48:46.505080 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b82d51-989f-4c86-85c8-09f0a3a68540 admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:46.506801 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-94b82d51-989f-4c86-85c8-09f0a3a68540 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:46.524113 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b82d51-989f-4c86-85c8-09f0a3a68540 admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:46.532019 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-2329cba2-3a53-460e-9de8-2964215e3751 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '50958e3a173847db998537e22c8f3efb', 'request_id': 'req-2329cba2-3a53-460e-9de8-2964215e3751'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:46.542331 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-2329cba2-3a53-460e-9de8-2964215e3751 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '50958e3a173847db998537e22c8f3efb', 'request_id': 'req-2329cba2-3a53-460e-9de8-2964215e3751'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:46.562317 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-2329cba2-3a53-460e-9de8-2964215e3751 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '50958e3a173847db998537e22c8f3efb', 'request_id': 'req-2329cba2-3a53-460e-9de8-2964215e3751'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:46.599070 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-2329cba2-3a53-460e-9de8-2964215e3751 admin admin] The token cache is being invalidate because project 50958e3a173847db998537e22c8f3efb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86827) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 10:48:46.599378 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-2329cba2-3a53-460e-9de8-2964215e3751 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:46.600066 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-2329cba2-3a53-460e-9de8-2964215e3751 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:46.606768 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1010/1988] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Jul 27 10:48:46 2022] DELETE /identity/v3/projects/50958e3a173847db998537e22c8f3efb => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:48:46.607627 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2329cba2-3a53-460e-9de8-2964215e3751 admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:46.609958 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-2329cba2-3a53-460e-9de8-2964215e3751 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:46.628286 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b5e893-edd8-4b11-a574-d772bd4614ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:46.629282 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b5e893-edd8-4b11-a574-d772bd4614ba admin admin] REQUEST_METHOD: `PUT` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:46.630814 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b5e893-edd8-4b11-a574-d772bd4614ba admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:46.631757 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b5e893-edd8-4b11-a574-d772bd4614ba admin admin] PATH_INFO: `/v3/projects/687ae3b37ca843aabd40a3218beb0092/users/5e9fec0a854b4a9cab8799dd51ed17ed/roles/4c9fc0b3baed441da2126be41cc6b04a` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:46.652850 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b5e893-edd8-4b11-a574-d772bd4614ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=687ae3b37ca843aabd40a3218beb0092, user_id=5e9fec0a854b4a9cab8799dd51ed17ed, role_id=4c9fc0b3baed441da2126be41cc6b04a)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:46.673084 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b5e893-edd8-4b11-a574-d772bd4614ba admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:46.698291 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 979/1989] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 10:48:46 2022] PUT /identity/v3/projects/687ae3b37ca843aabd40a3218beb0092/users/5e9fec0a854b4a9cab8799dd51ed17ed/roles/4c9fc0b3baed441da2126be41cc6b04a => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:48:46.700112 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b5e893-edd8-4b11-a574-d772bd4614ba admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:46.702705 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-b2b5e893-edd8-4b11-a574-d772bd4614ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:46.705826 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2329cba2-3a53-460e-9de8-2964215e3751 admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:46.770114 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:48:46.789414 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:48:46.799796 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b5e893-edd8-4b11-a574-d772bd4614ba admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:46.804645 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e30221-676d-4cbf-8c35-405b4baec827 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:46.805274 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e30221-676d-4cbf-8c35-405b4baec827 admin admin] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:46.805553 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e30221-676d-4cbf-8c35-405b4baec827 admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:46.805826 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e30221-676d-4cbf-8c35-405b4baec827 admin admin] PATH_INFO: `/v3/projects` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:46.806525 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e30221-676d-4cbf-8c35-405b4baec827 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:46.809012 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e30221-676d-4cbf-8c35-405b4baec827 admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:46.810817 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b48e59a-b680-497c-8d5e-52871be7ac3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:46.811453 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b48e59a-b680-497c-8d5e-52871be7ac3e admin admin] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:46.811736 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b48e59a-b680-497c-8d5e-52871be7ac3e admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:46.812013 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b48e59a-b680-497c-8d5e-52871be7ac3e admin admin] PATH_INFO: `/v3/projects` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:46.812682 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b48e59a-b680-497c-8d5e-52871be7ac3e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:46.814634 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b48e59a-b680-497c-8d5e-52871be7ac3e admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:46.836681 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 980/1990] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Jul 27 10:48:46 2022] POST /identity/v3/projects => generated 383 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 10:48:46.838078 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39528a89-1337-4e2b-a3fe-24e470db8416 None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:46.838418 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39528a89-1337-4e2b-a3fe-24e470db8416 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:46.838800 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39528a89-1337-4e2b-a3fe-24e470db8416 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:46.843324 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1011/1991] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Jul 27 10:48:46 2022] POST /identity/v3/projects => generated 377 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 10:48:46.844654 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b9a9c4-a0fc-4331-89f6-a824b4af8297 None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:46.844957 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b9a9c4-a0fc-4331-89f6-a824b4af8297 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:46.845232 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b9a9c4-a0fc-4331-89f6-a824b4af8297 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:46.860154 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-39528a89-1337-4e2b-a3fe-24e470db8416 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:48:46.861322 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-39528a89-1337-4e2b-a3fe-24e470db8416 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:46.875291 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-32b9a9c4-a0fc-4331-89f6-a824b4af8297 None None] MFA Rules not processed for user `5e9fec0a854b4a9cab8799dd51ed17ed`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:48:46.877028 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-32b9a9c4-a0fc-4331-89f6-a824b4af8297 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:46.930656 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 981/1992] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:48:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:48:46.931322 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39528a89-1337-4e2b-a3fe-24e470db8416 None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:46.944155 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-39528a89-1337-4e2b-a3fe-24e470db8416 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:46.962428 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39528a89-1337-4e2b-a3fe-24e470db8416 None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:46.966021 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1012/1993] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:48:46 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:48:46.966697 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b9a9c4-a0fc-4331-89f6-a824b4af8297 None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:46.970421 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c476ada-df42-4819-9381-d4787595ffad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:46.971120 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c476ada-df42-4819-9381-d4787595ffad admin admin] REQUEST_METHOD: `DELETE` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:46.971402 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c476ada-df42-4819-9381-d4787595ffad admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:46.971674 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c476ada-df42-4819-9381-d4787595ffad admin admin] PATH_INFO: `/v3/projects/6edec309c42a4ab080f9dc7bc061aefe` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:46.973001 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c476ada-df42-4819-9381-d4787595ffad admin admin] RBAC: Authorizing `identity:delete_project(project_id=6edec309c42a4ab080f9dc7bc061aefe)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:46.975050 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c476ada-df42-4819-9381-d4787595ffad admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:46.977307 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b9a9c4-a0fc-4331-89f6-a824b4af8297 None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:46.982861 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149db8d6-b135-480a-a342-4e1a70de153f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:46.983487 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149db8d6-b135-480a-a342-4e1a70de153f admin admin] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:46.983773 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149db8d6-b135-480a-a342-4e1a70de153f admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:46.984049 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149db8d6-b135-480a-a342-4e1a70de153f admin admin] PATH_INFO: `/v3/users` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:46.984719 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149db8d6-b135-480a-a342-4e1a70de153f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:46.986653 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149db8d6-b135-480a-a342-4e1a70de153f admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:47.011933 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1013/1994] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Jul 27 10:48:46 2022] POST /identity/v3/users => generated 292 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 10:48:47.012850 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149db8d6-b135-480a-a342-4e1a70de153f admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:47.022846 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149db8d6-b135-480a-a342-4e1a70de153f admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:47.032505 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-0c476ada-df42-4819-9381-d4787595ffad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6edec309c42a4ab080f9dc7bc061aefe', 'request_id': 'req-0c476ada-df42-4819-9381-d4787595ffad'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:47.040085 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-0c476ada-df42-4819-9381-d4787595ffad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6edec309c42a4ab080f9dc7bc061aefe', 'request_id': 'req-0c476ada-df42-4819-9381-d4787595ffad'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:47.055282 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-0c476ada-df42-4819-9381-d4787595ffad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6edec309c42a4ab080f9dc7bc061aefe', 'request_id': 'req-0c476ada-df42-4819-9381-d4787595ffad'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:47.069903 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-0c476ada-df42-4819-9381-d4787595ffad admin admin] The token cache is being invalidate because project 6edec309c42a4ab080f9dc7bc061aefe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86828) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 10:48:47.070429 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-0c476ada-df42-4819-9381-d4787595ffad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:47.071385 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-0c476ada-df42-4819-9381-d4787595ffad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:47.073169 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 982/1995] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Jul 27 10:48:46 2022] DELETE /identity/v3/projects/6edec309c42a4ab080f9dc7bc061aefe => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:48:47.074434 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea1cf6a-bfbe-4ddf-8c48-92407e68dd22 None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:47.074760 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea1cf6a-bfbe-4ddf-8c48-92407e68dd22 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:47.075234 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea1cf6a-bfbe-4ddf-8c48-92407e68dd22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:47.096081 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-1ea1cf6a-bfbe-4ddf-8c48-92407e68dd22 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:48:47.097367 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7fa83e-4d97-4bb4-8ff7-ca693ef6fff4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:47.097562 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-1ea1cf6a-bfbe-4ddf-8c48-92407e68dd22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:47.098130 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7fa83e-4d97-4bb4-8ff7-ca693ef6fff4 admin admin] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:47.098586 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7fa83e-4d97-4bb4-8ff7-ca693ef6fff4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:47.098966 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7fa83e-4d97-4bb4-8ff7-ca693ef6fff4 admin admin] PATH_INFO: `/v3/users` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:47.099851 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7fa83e-4d97-4bb4-8ff7-ca693ef6fff4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:47.101920 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7fa83e-4d97-4bb4-8ff7-ca693ef6fff4 admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:47.106768 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 983/1996] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:48:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:48:47.107628 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea1cf6a-bfbe-4ddf-8c48-92407e68dd22 None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:47.108897 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-1ea1cf6a-bfbe-4ddf-8c48-92407e68dd22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:47.123277 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea1cf6a-bfbe-4ddf-8c48-92407e68dd22 None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:47.130159 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1014/1997] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Jul 27 10:48:47 2022] POST /identity/v3/users => generated 289 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 10:48:47.131227 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7fa83e-4d97-4bb4-8ff7-ca693ef6fff4 admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:47.133677 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-bf7fa83e-4d97-4bb4-8ff7-ca693ef6fff4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:47.146147 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302687fd-aa03-4d91-a0b2-f199839b9537 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:47.148445 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302687fd-aa03-4d91-a0b2-f199839b9537 admin admin] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:47.148635 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7fa83e-4d97-4bb4-8ff7-ca693ef6fff4 admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:47.148757 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302687fd-aa03-4d91-a0b2-f199839b9537 admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:47.148999 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302687fd-aa03-4d91-a0b2-f199839b9537 admin admin] PATH_INFO: `/v3/projects` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:47.150980 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302687fd-aa03-4d91-a0b2-f199839b9537 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:47.155202 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302687fd-aa03-4d91-a0b2-f199839b9537 admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:47.155722 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d6069c-b8a3-4168-ab5c-eef6d4193425 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:47.156360 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d6069c-b8a3-4168-ab5c-eef6d4193425 admin admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:47.156644 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d6069c-b8a3-4168-ab5c-eef6d4193425 admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:47.158307 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d6069c-b8a3-4168-ab5c-eef6d4193425 admin admin] PATH_INFO: `/v3/roles` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:47.158307 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d6069c-b8a3-4168-ab5c-eef6d4193425 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:47.159751 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d6069c-b8a3-4168-ab5c-eef6d4193425 admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:47.172915 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1015/1998] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 10:48:47 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 10:48:47.174437 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9d547b-19a8-4aea-8943-b3e9b5b03959 None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:47.174774 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9d547b-19a8-4aea-8943-b3e9b5b03959 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:47.175056 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9d547b-19a8-4aea-8943-b3e9b5b03959 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:47.199100 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-4a9d547b-19a8-4aea-8943-b3e9b5b03959 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:48:47.200814 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-4a9d547b-19a8-4aea-8943-b3e9b5b03959 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:47.210943 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1016/1999] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:48:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:48:47.211929 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9d547b-19a8-4aea-8943-b3e9b5b03959 None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:47.218861 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 984/2000] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Jul 27 10:48:47 2022] POST /identity/v3/projects => generated 385 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 10:48:47.218861 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302687fd-aa03-4d91-a0b2-f199839b9537 admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:47.219342 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-302687fd-aa03-4d91-a0b2-f199839b9537 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:47.227495 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9d547b-19a8-4aea-8943-b3e9b5b03959 None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:47.244176 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:48:47.255834 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:48:47.311894 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302687fd-aa03-4d91-a0b2-f199839b9537 admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:47.318211 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47830bcd-40b0-402f-9343-84d2fe179ca7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:47.319334 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47830bcd-40b0-402f-9343-84d2fe179ca7 admin admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:47.319633 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47830bcd-40b0-402f-9343-84d2fe179ca7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:47.319916 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47830bcd-40b0-402f-9343-84d2fe179ca7 admin admin] PATH_INFO: `/v3/roles` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:47.320727 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47830bcd-40b0-402f-9343-84d2fe179ca7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:47.322684 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47830bcd-40b0-402f-9343-84d2fe179ca7 admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:47.329210 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e6b5d8-94e3-410e-a2b8-274825a1d821 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:47.331123 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e6b5d8-94e3-410e-a2b8-274825a1d821 admin admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:47.331773 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e6b5d8-94e3-410e-a2b8-274825a1d821 admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:47.332250 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e6b5d8-94e3-410e-a2b8-274825a1d821 admin admin] PATH_INFO: `/v3/domains` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:47.333190 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e6b5d8-94e3-410e-a2b8-274825a1d821 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86827) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:48:47.333736 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e6b5d8-94e3-410e-a2b8-274825a1d821 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:47.335723 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e6b5d8-94e3-410e-a2b8-274825a1d821 admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:47.340161 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 985/2001] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 10:48:47 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 10:48:47.341084 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47830bcd-40b0-402f-9343-84d2fe179ca7 admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:47.353208 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1017/2002] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Jul 27 10:48:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:48:47.353363 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47830bcd-40b0-402f-9343-84d2fe179ca7 admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:47.354623 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ad3467-7798-4a1a-867b-6d7066bb987a None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:47.355007 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ad3467-7798-4a1a-867b-6d7066bb987a None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:47.356050 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ad3467-7798-4a1a-867b-6d7066bb987a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:47.359994 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c661928-7a14-47ca-b0d1-16efd72ca310 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:47.361213 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c661928-7a14-47ca-b0d1-16efd72ca310 admin admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:47.361213 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c661928-7a14-47ca-b0d1-16efd72ca310 admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:47.361213 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c661928-7a14-47ca-b0d1-16efd72ca310 admin admin] PATH_INFO: `/v3/roles` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:47.361890 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c661928-7a14-47ca-b0d1-16efd72ca310 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:47.363595 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c661928-7a14-47ca-b0d1-16efd72ca310 admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:47.378198 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 986/2003] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 10:48:47 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 10:48:47.378958 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c661928-7a14-47ca-b0d1-16efd72ca310 admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:47.383270 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-34ad3467-7798-4a1a-867b-6d7066bb987a None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:48:47.384562 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-34ad3467-7798-4a1a-867b-6d7066bb987a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:47.392450 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c661928-7a14-47ca-b0d1-16efd72ca310 admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:47.394291 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1018/2004] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:48:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:48:47.398144 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ad3467-7798-4a1a-867b-6d7066bb987a None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:47.400686 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408bbc1d-f1be-413f-89d3-71f2814d6c8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:47.401633 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408bbc1d-f1be-413f-89d3-71f2814d6c8f admin admin] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:47.402816 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408bbc1d-f1be-413f-89d3-71f2814d6c8f admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:47.403310 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408bbc1d-f1be-413f-89d3-71f2814d6c8f admin admin] PATH_INFO: `/v3/users` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:47.404137 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408bbc1d-f1be-413f-89d3-71f2814d6c8f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:47.407580 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408bbc1d-f1be-413f-89d3-71f2814d6c8f admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:47.412407 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ad3467-7798-4a1a-867b-6d7066bb987a None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:47.419168 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9d5bb4-a69d-45bc-9416-ebf201ec98f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:47.419905 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9d5bb4-a69d-45bc-9416-ebf201ec98f0 admin admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:47.420451 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9d5bb4-a69d-45bc-9416-ebf201ec98f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:47.420785 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9d5bb4-a69d-45bc-9416-ebf201ec98f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:47.421542 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9d5bb4-a69d-45bc-9416-ebf201ec98f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:47.423812 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9d5bb4-a69d-45bc-9416-ebf201ec98f0 admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:47.438273 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 987/2005] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Jul 27 10:48:47 2022] POST /identity/v3/users => generated 299 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 10:48:47.439207 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408bbc1d-f1be-413f-89d3-71f2814d6c8f admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:47.452138 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408bbc1d-f1be-413f-89d3-71f2814d6c8f admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:47.458163 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1019/2006] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 10:48:47 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 10:48:47.461463 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9d5bb4-a69d-45bc-9416-ebf201ec98f0 admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:47.471051 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb418829-eeab-40c9-8d2a-5f0b230587a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:47.472072 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb418829-eeab-40c9-8d2a-5f0b230587a1 admin admin] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:47.472251 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9d5bb4-a69d-45bc-9416-ebf201ec98f0 admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:47.472715 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb418829-eeab-40c9-8d2a-5f0b230587a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:47.473388 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb418829-eeab-40c9-8d2a-5f0b230587a1 admin admin] PATH_INFO: `/v3/projects` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:47.474961 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb418829-eeab-40c9-8d2a-5f0b230587a1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:47.477177 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c98e3e1-d7e9-4fce-b0c8-58ffdc4e788d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:47.477849 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c98e3e1-d7e9-4fce-b0c8-58ffdc4e788d admin admin] REQUEST_METHOD: `PUT` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:47.477958 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb418829-eeab-40c9-8d2a-5f0b230587a1 admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:47.478132 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c98e3e1-d7e9-4fce-b0c8-58ffdc4e788d admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:47.478415 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c98e3e1-d7e9-4fce-b0c8-58ffdc4e788d admin admin] PATH_INFO: `/v3/projects/7ca07b06b6a94d30a8e58ec294519ee9/users/cc81658ff1bc48c1b5b4cfc1d19ee9fe/roles/4c9fc0b3baed441da2126be41cc6b04a` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:47.498555 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c98e3e1-d7e9-4fce-b0c8-58ffdc4e788d admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ca07b06b6a94d30a8e58ec294519ee9, user_id=cc81658ff1bc48c1b5b4cfc1d19ee9fe, role_id=4c9fc0b3baed441da2126be41cc6b04a)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:47.514937 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c98e3e1-d7e9-4fce-b0c8-58ffdc4e788d admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:47.526636 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 988/2007] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Jul 27 10:48:47 2022] POST /identity/v3/projects => generated 377 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 10:48:47.528177 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb418829-eeab-40c9-8d2a-5f0b230587a1 admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:47.534086 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1020/2008] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 10:48:47 2022] PUT /identity/v3/projects/7ca07b06b6a94d30a8e58ec294519ee9/users/cc81658ff1bc48c1b5b4cfc1d19ee9fe/roles/4c9fc0b3baed441da2126be41cc6b04a => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:48:47.535560 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c98e3e1-d7e9-4fce-b0c8-58ffdc4e788d admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:47.600691 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb418829-eeab-40c9-8d2a-5f0b230587a1 admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:47.605799 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f427a11a-7ef5-44c2-a9cd-8b1204e15183 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:47.606436 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f427a11a-7ef5-44c2-a9cd-8b1204e15183 admin admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:47.606782 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f427a11a-7ef5-44c2-a9cd-8b1204e15183 admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:47.607065 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f427a11a-7ef5-44c2-a9cd-8b1204e15183 admin admin] PATH_INFO: `/v3/domains` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:47.607798 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f427a11a-7ef5-44c2-a9cd-8b1204e15183 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86828) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:48:47.608105 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f427a11a-7ef5-44c2-a9cd-8b1204e15183 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:47.609714 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f427a11a-7ef5-44c2-a9cd-8b1204e15183 admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:47.615572 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c98e3e1-d7e9-4fce-b0c8-58ffdc4e788d admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:47.620295 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2433081f-ef60-492a-b511-fe56f0e71a09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:47.620927 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2433081f-ef60-492a-b511-fe56f0e71a09 admin admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:47.621256 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2433081f-ef60-492a-b511-fe56f0e71a09 admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:47.621559 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2433081f-ef60-492a-b511-fe56f0e71a09 admin admin] PATH_INFO: `/v3/roles` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:47.622310 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2433081f-ef60-492a-b511-fe56f0e71a09 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:47.623337 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 989/2009] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Jul 27 10:48:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:48:47.623971 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2433081f-ef60-492a-b511-fe56f0e71a09 admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:47.624139 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f427a11a-7ef5-44c2-a9cd-8b1204e15183 admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:47.638018 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1021/2010] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 10:48:47 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 10:48:47.638215 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f427a11a-7ef5-44c2-a9cd-8b1204e15183 admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:47.638315 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2433081f-ef60-492a-b511-fe56f0e71a09 admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:47.640086 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee40b6b-bdd6-4103-b897-f3e7bdf81022 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:47.640751 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee40b6b-bdd6-4103-b897-f3e7bdf81022 admin admin] REQUEST_METHOD: `PUT` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:47.641052 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee40b6b-bdd6-4103-b897-f3e7bdf81022 admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:47.641314 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee40b6b-bdd6-4103-b897-f3e7bdf81022 admin admin] PATH_INFO: `/v3/projects/d10675b447774d24940bd500f20ffba2/users/c5c353bffcb143f59e9c0936cd7c4cdf/roles/4c9fc0b3baed441da2126be41cc6b04a` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:47.646383 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2433081f-ef60-492a-b511-fe56f0e71a09 admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:47.651810 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03791e69-3712-4ae6-ae67-940b09d3a861 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:47.653002 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03791e69-3712-4ae6-ae67-940b09d3a861 admin admin] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:47.653363 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03791e69-3712-4ae6-ae67-940b09d3a861 admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:47.653636 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03791e69-3712-4ae6-ae67-940b09d3a861 admin admin] PATH_INFO: `/v3/users` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:47.654293 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03791e69-3712-4ae6-ae67-940b09d3a861 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:47.656225 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03791e69-3712-4ae6-ae67-940b09d3a861 admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:47.657729 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee40b6b-bdd6-4103-b897-f3e7bdf81022 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d10675b447774d24940bd500f20ffba2, user_id=c5c353bffcb143f59e9c0936cd7c4cdf, role_id=4c9fc0b3baed441da2126be41cc6b04a)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:47.660359 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee40b6b-bdd6-4103-b897-f3e7bdf81022 admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:47.675106 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 990/2011] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 10:48:47 2022] PUT /identity/v3/projects/d10675b447774d24940bd500f20ffba2/users/c5c353bffcb143f59e9c0936cd7c4cdf/roles/4c9fc0b3baed441da2126be41cc6b04a => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:48:47.685219 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1fb5fb-f9a6-4e75-9d59-750fdee6dd10 None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:47.685219 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1fb5fb-f9a6-4e75-9d59-750fdee6dd10 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:47.685219 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1fb5fb-f9a6-4e75-9d59-750fdee6dd10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:47.685511 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1022/2012] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Jul 27 10:48:47 2022] POST /identity/v3/users => generated 289 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 10:48:47.685511 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03791e69-3712-4ae6-ae67-940b09d3a861 admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:47.690509 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03791e69-3712-4ae6-ae67-940b09d3a861 admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:47.705549 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-be1fb5fb-f9a6-4e75-9d59-750fdee6dd10 None None] MFA Rules not processed for user `cc81658ff1bc48c1b5b4cfc1d19ee9fe`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:48:47.706785 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-be1fb5fb-f9a6-4e75-9d59-750fdee6dd10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:47.735580 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:48:47.737069 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:48:47.754967 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75659209-0d26-42c2-99c2-062a89d538b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:47.755606 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75659209-0d26-42c2-99c2-062a89d538b8 admin admin] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:47.755893 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75659209-0d26-42c2-99c2-062a89d538b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:47.756173 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75659209-0d26-42c2-99c2-062a89d538b8 admin admin] PATH_INFO: `/v3/projects` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:47.756846 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75659209-0d26-42c2-99c2-062a89d538b8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:47.758955 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75659209-0d26-42c2-99c2-062a89d538b8 admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:47.785818 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 991/2013] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:48:47 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:48:47.786805 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1fb5fb-f9a6-4e75-9d59-750fdee6dd10 None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:47.791608 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1023/2014] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Jul 27 10:48:47 2022] POST /identity/v3/projects => generated 385 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 10:48:47.792734 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75659209-0d26-42c2-99c2-062a89d538b8 admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:47.802122 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1fb5fb-f9a6-4e75-9d59-750fdee6dd10 None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:47.869242 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75659209-0d26-42c2-99c2-062a89d538b8 admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:47.876058 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0023f1-5d24-455a-8ef7-6047adcface1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:47.879478 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0023f1-5d24-455a-8ef7-6047adcface1 admin admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:47.879774 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0023f1-5d24-455a-8ef7-6047adcface1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:47.880061 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0023f1-5d24-455a-8ef7-6047adcface1 admin admin] PATH_INFO: `/v3/roles` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:47.880762 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0023f1-5d24-455a-8ef7-6047adcface1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:47.882102 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8be6b1c-feaa-4fa8-abfd-27bec9567335 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:47.882440 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0023f1-5d24-455a-8ef7-6047adcface1 admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:47.883020 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8be6b1c-feaa-4fa8-abfd-27bec9567335 admin admin] REQUEST_METHOD: `PUT` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:47.883308 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8be6b1c-feaa-4fa8-abfd-27bec9567335 admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:47.883590 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8be6b1c-feaa-4fa8-abfd-27bec9567335 admin admin] PATH_INFO: `/v3/projects/ecdaa8585aaa41a09c7938e3b30a6040/users/b6bc7f4437474795958c10e50c0d4e55/roles/21156560eb6b436bb567a40b4976f3f4` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:47.896877 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1024/2015] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 10:48:47 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 10:48:47.896877 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19aa6dd8-a2f1-4c05-936b-cf1e2449d3d7 None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:47.896877 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19aa6dd8-a2f1-4c05-936b-cf1e2449d3d7 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:47.896877 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19aa6dd8-a2f1-4c05-936b-cf1e2449d3d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:47.900494 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8be6b1c-feaa-4fa8-abfd-27bec9567335 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ecdaa8585aaa41a09c7938e3b30a6040, user_id=b6bc7f4437474795958c10e50c0d4e55, role_id=21156560eb6b436bb567a40b4976f3f4)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:47.903305 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8be6b1c-feaa-4fa8-abfd-27bec9567335 admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:47.917948 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 992/2016] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 10:48:47 2022] PUT /identity/v3/projects/ecdaa8585aaa41a09c7938e3b30a6040/users/b6bc7f4437474795958c10e50c0d4e55/roles/21156560eb6b436bb567a40b4976f3f4 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:48:47.921757 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8be6b1c-feaa-4fa8-abfd-27bec9567335 admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:47.930322 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-19aa6dd8-a2f1-4c05-936b-cf1e2449d3d7 None None] MFA Rules not processed for user `c5c353bffcb143f59e9c0936cd7c4cdf`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:48:47.931598 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-19aa6dd8-a2f1-4c05-936b-cf1e2449d3d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:47.995436 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8be6b1c-feaa-4fa8-abfd-27bec9567335 admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:48.001320 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f062c93f-6e51-4fd4-b7de-57b144cf154c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:48.002645 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f062c93f-6e51-4fd4-b7de-57b144cf154c admin admin] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:48.002645 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f062c93f-6e51-4fd4-b7de-57b144cf154c admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:48.002908 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f062c93f-6e51-4fd4-b7de-57b144cf154c admin admin] PATH_INFO: `/v3/projects` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:48.003722 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f062c93f-6e51-4fd4-b7de-57b144cf154c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:48.005892 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f062c93f-6e51-4fd4-b7de-57b144cf154c admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:48.036720 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 993/2017] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Jul 27 10:48:47 2022] POST /identity/v3/projects => generated 383 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 10:48:48.037640 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f062c93f-6e51-4fd4-b7de-57b144cf154c admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:48.052713 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1025/2018] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:48:47 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:48:48.054303 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19aa6dd8-a2f1-4c05-936b-cf1e2449d3d7 None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:48.065887 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19aa6dd8-a2f1-4c05-936b-cf1e2449d3d7 None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:48.087815 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:48:48.114094 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f062c93f-6e51-4fd4-b7de-57b144cf154c admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:48.119129 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1ddebd-e6a5-48b5-9850-24076afa36d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:48.119703 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1ddebd-e6a5-48b5-9850-24076afa36d2 admin admin] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:48.119992 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1ddebd-e6a5-48b5-9850-24076afa36d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:48.120273 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1ddebd-e6a5-48b5-9850-24076afa36d2 admin admin] PATH_INFO: `/v3/users` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:48.121498 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1ddebd-e6a5-48b5-9850-24076afa36d2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:48.123842 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1ddebd-e6a5-48b5-9850-24076afa36d2 admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:48.128493 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89895df5-1e08-42c3-a071-4636dcaa3699 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:48.129300 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89895df5-1e08-42c3-a071-4636dcaa3699 admin admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:48.129654 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89895df5-1e08-42c3-a071-4636dcaa3699 admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:48.130107 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89895df5-1e08-42c3-a071-4636dcaa3699 admin admin] PATH_INFO: `/v3/roles` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:48.130924 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89895df5-1e08-42c3-a071-4636dcaa3699 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:48.132781 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89895df5-1e08-42c3-a071-4636dcaa3699 admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:48.134980 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:48:48.147301 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1026/2019] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 10:48:48 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 10:48:48.148178 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89895df5-1e08-42c3-a071-4636dcaa3699 admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:48.150717 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 994/2020] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Jul 27 10:48:48 2022] POST /identity/v3/users => generated 293 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 10:48:48.151911 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1ddebd-e6a5-48b5-9850-24076afa36d2 admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:48.164181 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89895df5-1e08-42c3-a071-4636dcaa3699 admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:48.167508 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1ddebd-e6a5-48b5-9850-24076afa36d2 admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:48.169225 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782ad9f0-97b4-41a3-bf2e-614c7b697e86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:48.169847 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782ad9f0-97b4-41a3-bf2e-614c7b697e86 admin admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:48.170132 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782ad9f0-97b4-41a3-bf2e-614c7b697e86 admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:48.170409 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782ad9f0-97b4-41a3-bf2e-614c7b697e86 admin admin] PATH_INFO: `/v3/roles` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:48.171138 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782ad9f0-97b4-41a3-bf2e-614c7b697e86 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:48.172552 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66a2f0f-ea00-4977-b443-017a5604bc3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:48.172738 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782ad9f0-97b4-41a3-bf2e-614c7b697e86 admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:48.173333 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66a2f0f-ea00-4977-b443-017a5604bc3b admin admin] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:48.173717 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66a2f0f-ea00-4977-b443-017a5604bc3b admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:48.174069 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66a2f0f-ea00-4977-b443-017a5604bc3b admin admin] PATH_INFO: `/v3/users` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:48.174840 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66a2f0f-ea00-4977-b443-017a5604bc3b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:48.177055 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66a2f0f-ea00-4977-b443-017a5604bc3b admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:48.184262 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1027/2021] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 10:48:48 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 10:48:48.184951 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782ad9f0-97b4-41a3-bf2e-614c7b697e86 admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:48.196017 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782ad9f0-97b4-41a3-bf2e-614c7b697e86 admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:48.201357 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4a7a25-a7f7-4b9e-9c63-ec25c9b778e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:48.201973 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4a7a25-a7f7-4b9e-9c63-ec25c9b778e4 admin admin] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:48.202262 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4a7a25-a7f7-4b9e-9c63-ec25c9b778e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:48.202541 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4a7a25-a7f7-4b9e-9c63-ec25c9b778e4 admin admin] PATH_INFO: `/v3/projects` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:48.203241 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4a7a25-a7f7-4b9e-9c63-ec25c9b778e4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:48.205142 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4a7a25-a7f7-4b9e-9c63-ec25c9b778e4 admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:48.213943 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 995/2022] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Jul 27 10:48:48 2022] POST /identity/v3/users => generated 298 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 10:48:48.215121 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66a2f0f-ea00-4977-b443-017a5604bc3b admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:48.224827 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66a2f0f-ea00-4977-b443-017a5604bc3b admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:48.229979 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268099ff-d589-4a11-8e4a-203f8448eca2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:48.231183 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268099ff-d589-4a11-8e4a-203f8448eca2 admin admin] REQUEST_METHOD: `PUT` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:48.231464 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268099ff-d589-4a11-8e4a-203f8448eca2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:48.231739 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268099ff-d589-4a11-8e4a-203f8448eca2 admin admin] PATH_INFO: `/v3/projects/3df1b61ddbb849aaaac33e3b04ffca32/users/d80b5a817afd48ec9fb87b63b8d85e9a/roles/4c9fc0b3baed441da2126be41cc6b04a` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:48.238776 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1028/2023] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Jul 27 10:48:48 2022] POST /identity/v3/projects => generated 377 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 10:48:48.239805 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4a7a25-a7f7-4b9e-9c63-ec25c9b778e4 admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:48.248442 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268099ff-d589-4a11-8e4a-203f8448eca2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3df1b61ddbb849aaaac33e3b04ffca32, user_id=d80b5a817afd48ec9fb87b63b8d85e9a, role_id=4c9fc0b3baed441da2126be41cc6b04a)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:48.251112 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268099ff-d589-4a11-8e4a-203f8448eca2 admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:48.266313 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 996/2024] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 10:48:48 2022] PUT /identity/v3/projects/3df1b61ddbb849aaaac33e3b04ffca32/users/d80b5a817afd48ec9fb87b63b8d85e9a/roles/4c9fc0b3baed441da2126be41cc6b04a => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:48:48.267228 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268099ff-d589-4a11-8e4a-203f8448eca2 admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:48.314489 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4a7a25-a7f7-4b9e-9c63-ec25c9b778e4 admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:48.346110 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268099ff-d589-4a11-8e4a-203f8448eca2 admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:48.351726 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e69bac9-9fe0-4361-a82a-1c076912258e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:48.352417 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e69bac9-9fe0-4361-a82a-1c076912258e admin admin] REQUEST_METHOD: `PUT` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:48.352705 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e69bac9-9fe0-4361-a82a-1c076912258e admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:48.352984 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e69bac9-9fe0-4361-a82a-1c076912258e admin admin] PATH_INFO: `/v3/projects/ecdaa8585aaa41a09c7938e3b30a6040/users/b6bc7f4437474795958c10e50c0d4e55/roles/21156560eb6b436bb567a40b4976f3f4` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:48.355001 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e69bac9-9fe0-4361-a82a-1c076912258e admin admin] RBAC: Authorizing `identity:create_grant(project_id=ecdaa8585aaa41a09c7938e3b30a6040, user_id=b6bc7f4437474795958c10e50c0d4e55, role_id=21156560eb6b436bb567a40b4976f3f4)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:48.358824 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e69bac9-9fe0-4361-a82a-1c076912258e admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:48.369820 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 997/2025] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 10:48:48 2022] PUT /identity/v3/projects/ecdaa8585aaa41a09c7938e3b30a6040/users/b6bc7f4437474795958c10e50c0d4e55/roles/21156560eb6b436bb567a40b4976f3f4 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:48:48.370558 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e69bac9-9fe0-4361-a82a-1c076912258e admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:48.379926 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970de166-3d30-4ab9-b2c6-8320ea4e5fb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:48.380573 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970de166-3d30-4ab9-b2c6-8320ea4e5fb0 admin admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:48.381549 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970de166-3d30-4ab9-b2c6-8320ea4e5fb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:48.381549 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970de166-3d30-4ab9-b2c6-8320ea4e5fb0 admin admin] PATH_INFO: `/v3/roles` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:48.381991 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970de166-3d30-4ab9-b2c6-8320ea4e5fb0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:48.383892 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970de166-3d30-4ab9-b2c6-8320ea4e5fb0 admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:48.398055 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1029/2026] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 10:48:48 2022] GET /identity/v3/roles => generated 1420 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 10:48:48.398055 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970de166-3d30-4ab9-b2c6-8320ea4e5fb0 admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:48.410034 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970de166-3d30-4ab9-b2c6-8320ea4e5fb0 admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:48.449743 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e69bac9-9fe0-4361-a82a-1c076912258e admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:48.454874 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cf4d5e-9e7e-4cf8-8f2a-fe1b465bd184 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:48.455149 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cf4d5e-9e7e-4cf8-8f2a-fe1b465bd184 admin admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:48.455432 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cf4d5e-9e7e-4cf8-8f2a-fe1b465bd184 admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:48.455702 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cf4d5e-9e7e-4cf8-8f2a-fe1b465bd184 admin admin] PATH_INFO: `/v3/roles` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:48.456401 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43cf4d5e-9e7e-4cf8-8f2a-fe1b465bd184 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:48.458001 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43cf4d5e-9e7e-4cf8-8f2a-fe1b465bd184 admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:48.468391 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 998/2027] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 10:48:48 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 10:48:48.469420 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948b3067-8fd4-400f-83d4-4e2212c06f22 None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:48.469707 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948b3067-8fd4-400f-83d4-4e2212c06f22 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:48.469984 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948b3067-8fd4-400f-83d4-4e2212c06f22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:48.478810 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdc7011-33ec-459e-be92-a80ff504168a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:48.479676 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdc7011-33ec-459e-be92-a80ff504168a admin admin] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:48.480816 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdc7011-33ec-459e-be92-a80ff504168a admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:48.481195 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdc7011-33ec-459e-be92-a80ff504168a admin admin] PATH_INFO: `/v3/users` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:48.481979 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cdc7011-33ec-459e-be92-a80ff504168a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:48.484166 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cdc7011-33ec-459e-be92-a80ff504168a admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:48.495814 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:48:48.501823 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-948b3067-8fd4-400f-83d4-4e2212c06f22 None None] MFA Rules not processed for user `d80b5a817afd48ec9fb87b63b8d85e9a`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:48:48.503212 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-948b3067-8fd4-400f-83d4-4e2212c06f22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:48.507726 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:48:48.514891 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1030/2028] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Jul 27 10:48:48 2022] POST /identity/v3/users => generated 295 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 10:48:48.514891 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed51eae4-f8fe-4461-b126-e5b9c56c8fef None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:48.514891 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed51eae4-f8fe-4461-b126-e5b9c56c8fef None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:48.514891 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed51eae4-f8fe-4461-b126-e5b9c56c8fef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:48.547911 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-ed51eae4-f8fe-4461-b126-e5b9c56c8fef None None] MFA Rules not processed for user `b6bc7f4437474795958c10e50c0d4e55`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:48:48.549164 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-ed51eae4-f8fe-4461-b126-e5b9c56c8fef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:48.589963 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 999/2029] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:48:48 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:48:48.591338 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948b3067-8fd4-400f-83d4-4e2212c06f22 None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:48.617631 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948b3067-8fd4-400f-83d4-4e2212c06f22 None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:48.628607 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b39193b-19e8-432c-93c3-d7e77380913a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:48.629176 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b39193b-19e8-432c-93c3-d7e77380913a admin admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:48.629461 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b39193b-19e8-432c-93c3-d7e77380913a admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:48.629737 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b39193b-19e8-432c-93c3-d7e77380913a admin admin] PATH_INFO: `/v3/roles` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:48.630446 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b39193b-19e8-432c-93c3-d7e77380913a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:48.632112 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b39193b-19e8-432c-93c3-d7e77380913a admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:48.644713 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1000/2030] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 10:48:48 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 10:48:48.645654 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b39193b-19e8-432c-93c3-d7e77380913a admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:48.654007 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1031/2031] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:48:48 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:48:48.655197 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed51eae4-f8fe-4461-b126-e5b9c56c8fef None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:48.668761 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b39193b-19e8-432c-93c3-d7e77380913a admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:48.669884 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed51eae4-f8fe-4461-b126-e5b9c56c8fef None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:48.681551 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8b009e-d47c-44d5-85e9-1d042bb0193d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:48.682267 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8b009e-d47c-44d5-85e9-1d042bb0193d admin admin] REQUEST_METHOD: `PUT` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:48.682546 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8b009e-d47c-44d5-85e9-1d042bb0193d admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:48.683276 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8b009e-d47c-44d5-85e9-1d042bb0193d admin admin] PATH_INFO: `/v3/projects/e62fdabd05bc4fdc96e74c1d09d3ae11/users/82ad852314de4870ba9b3626051ad0c6/roles/21156560eb6b436bb567a40b4976f3f4` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:48.685519 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce9c93c-3cfb-4fc7-9f5e-a8b7e115ec60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:48.686396 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce9c93c-3cfb-4fc7-9f5e-a8b7e115ec60 admin admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:48.686810 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce9c93c-3cfb-4fc7-9f5e-a8b7e115ec60 admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:48.687169 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce9c93c-3cfb-4fc7-9f5e-a8b7e115ec60 admin admin] PATH_INFO: `/v3/roles` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:48.687960 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce9c93c-3cfb-4fc7-9f5e-a8b7e115ec60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:48.689582 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce9c93c-3cfb-4fc7-9f5e-a8b7e115ec60 admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:48.703197 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1032/2032] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 10:48:48 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 10:48:48.703874 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8b009e-d47c-44d5-85e9-1d042bb0193d admin admin] RBAC: Authorizing `identity:create_grant(project_id=e62fdabd05bc4fdc96e74c1d09d3ae11, user_id=82ad852314de4870ba9b3626051ad0c6, role_id=21156560eb6b436bb567a40b4976f3f4)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:48.704363 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce9c93c-3cfb-4fc7-9f5e-a8b7e115ec60 admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:48.706209 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-bce9c93c-3cfb-4fc7-9f5e-a8b7e115ec60 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:48.708042 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8b009e-d47c-44d5-85e9-1d042bb0193d admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:48.731782 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1001/2033] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 10:48:48 2022] PUT /identity/v3/projects/e62fdabd05bc4fdc96e74c1d09d3ae11/users/82ad852314de4870ba9b3626051ad0c6/roles/21156560eb6b436bb567a40b4976f3f4 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:48:48.732954 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8b009e-d47c-44d5-85e9-1d042bb0193d admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:48.798179 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce9c93c-3cfb-4fc7-9f5e-a8b7e115ec60 admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:48.811993 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8b009e-d47c-44d5-85e9-1d042bb0193d admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:48.817128 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d219b87-da6f-43bc-92d7-2b63eb040c88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:48.818062 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d219b87-da6f-43bc-92d7-2b63eb040c88 admin admin] REQUEST_METHOD: `PUT` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:48.818476 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d219b87-da6f-43bc-92d7-2b63eb040c88 admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:48.818886 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d219b87-da6f-43bc-92d7-2b63eb040c88 admin admin] PATH_INFO: `/v3/projects/33ddf88108e94513b7d66246ff7315cd/users/a23c4f92439442c5a70ea683ff2ca0bc/roles/4c9fc0b3baed441da2126be41cc6b04a` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:48.836894 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d219b87-da6f-43bc-92d7-2b63eb040c88 admin admin] RBAC: Authorizing `identity:create_grant(project_id=33ddf88108e94513b7d66246ff7315cd, user_id=a23c4f92439442c5a70ea683ff2ca0bc, role_id=4c9fc0b3baed441da2126be41cc6b04a)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:48.840266 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d219b87-da6f-43bc-92d7-2b63eb040c88 admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:48.858209 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1002/2034] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 10:48:48 2022] PUT /identity/v3/projects/33ddf88108e94513b7d66246ff7315cd/users/a23c4f92439442c5a70ea683ff2ca0bc/roles/4c9fc0b3baed441da2126be41cc6b04a => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:48:48.860159 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d219b87-da6f-43bc-92d7-2b63eb040c88 admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:48.867486 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a56f25-709c-4ce7-b187-b3c8663e9ff5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '864d35f268cf45148756109759bb2743', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:48.869835 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a56f25-709c-4ce7-b187-b3c8663e9ff5 service cinder] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:48.870449 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a56f25-709c-4ce7-b187-b3c8663e9ff5 service cinder] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:48.870918 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a56f25-709c-4ce7-b187-b3c8663e9ff5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:48.925125 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:48:48.925417 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:48:48.941159 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d219b87-da6f-43bc-92d7-2b63eb040c88 admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:48.950431 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20221ed7-14ec-401a-b5a8-f87e22db9dc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:48.953300 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20221ed7-14ec-401a-b5a8-f87e22db9dc6 admin admin] REQUEST_METHOD: `PUT` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:48.953759 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20221ed7-14ec-401a-b5a8-f87e22db9dc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:48.954194 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20221ed7-14ec-401a-b5a8-f87e22db9dc6 admin admin] PATH_INFO: `/v3/projects/21c10e481b60469da940254d0ea10fb4/users/ccd048bb2e0847ada5f6aed923672dea/roles/21156560eb6b436bb567a40b4976f3f4` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:48.954976 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a56f25-709c-4ce7-b187-b3c8663e9ff5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:48.957494 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a56f25-709c-4ce7-b187-b3c8663e9ff5 service cinder] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:48.974204 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1033/2035] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:48:48 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 270 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 10:48:48.975750 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a56f25-709c-4ce7-b187-b3c8663e9ff5 service cinder] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:48.977603 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-61a56f25-709c-4ce7-b187-b3c8663e9ff5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:48.977862 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20221ed7-14ec-401a-b5a8-f87e22db9dc6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=21c10e481b60469da940254d0ea10fb4, user_id=ccd048bb2e0847ada5f6aed923672dea, role_id=21156560eb6b436bb567a40b4976f3f4)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:48.981818 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20221ed7-14ec-401a-b5a8-f87e22db9dc6 admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:48.997232 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a56f25-709c-4ce7-b187-b3c8663e9ff5 service cinder] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:49.002913 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1003/2036] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 10:48:48 2022] PUT /identity/v3/projects/21c10e481b60469da940254d0ea10fb4/users/ccd048bb2e0847ada5f6aed923672dea/roles/21156560eb6b436bb567a40b4976f3f4 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:48:49.004600 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20221ed7-14ec-401a-b5a8-f87e22db9dc6 admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:49.064719 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303fd100-deba-4537-a878-1ace80101838 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '864d35f268cf45148756109759bb2743', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:49.065359 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303fd100-deba-4537-a878-1ace80101838 service cinder] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:49.065642 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303fd100-deba-4537-a878-1ace80101838 service cinder] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:49.065921 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303fd100-deba-4537-a878-1ace80101838 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:49.068040 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-303fd100-deba-4537-a878-1ace80101838 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:49.136085 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20221ed7-14ec-401a-b5a8-f87e22db9dc6 admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:49.141228 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303fd100-deba-4537-a878-1ace80101838 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:49.143147 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303fd100-deba-4537-a878-1ace80101838 service cinder] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:49.150828 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02522769-5962-447f-9c81-0106b1cb3c6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:49.153390 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02522769-5962-447f-9c81-0106b1cb3c6a admin admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:49.153762 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02522769-5962-447f-9c81-0106b1cb3c6a admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:49.153990 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02522769-5962-447f-9c81-0106b1cb3c6a admin admin] PATH_INFO: `/v3/roles` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:49.158013 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02522769-5962-447f-9c81-0106b1cb3c6a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:49.160289 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02522769-5962-447f-9c81-0106b1cb3c6a admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:49.165776 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1034/2037] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:48:48 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 10:48:49.167378 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e138fa8-b58f-4c28-822b-d4050a8d4c56 None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:49.167743 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e138fa8-b58f-4c28-822b-d4050a8d4c56 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:49.167978 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e138fa8-b58f-4c28-822b-d4050a8d4c56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:49.176300 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1004/2038] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 10:48:49 2022] GET /identity/v3/roles => generated 1420 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 10:48:49.179048 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02522769-5962-447f-9c81-0106b1cb3c6a admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:49.193088 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02522769-5962-447f-9c81-0106b1cb3c6a admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:49.197761 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f940bde2-56d1-439f-9771-fde09247119c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:49.198747 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f940bde2-56d1-439f-9771-fde09247119c admin admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:49.198747 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f940bde2-56d1-439f-9771-fde09247119c admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:49.199556 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f940bde2-56d1-439f-9771-fde09247119c admin admin] PATH_INFO: `/v3/roles` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:49.200226 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f940bde2-56d1-439f-9771-fde09247119c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:49.201831 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f940bde2-56d1-439f-9771-fde09247119c admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:49.215494 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1005/2039] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 10:48:49 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 10:48:49.215952 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f940bde2-56d1-439f-9771-fde09247119c admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:49.219106 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-3e138fa8-b58f-4c28-822b-d4050a8d4c56 None None] MFA Rules not processed for user `a23c4f92439442c5a70ea683ff2ca0bc`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:48:49.220596 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-3e138fa8-b58f-4c28-822b-d4050a8d4c56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:49.230137 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f940bde2-56d1-439f-9771-fde09247119c admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:49.238479 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e350cb-247e-415b-a558-5228cf7cb378 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:49.239273 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e350cb-247e-415b-a558-5228cf7cb378 admin admin] REQUEST_METHOD: `PUT` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:49.239637 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e350cb-247e-415b-a558-5228cf7cb378 admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:49.239969 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e350cb-247e-415b-a558-5228cf7cb378 admin admin] PATH_INFO: `/v3/projects/e62fdabd05bc4fdc96e74c1d09d3ae11/users/82ad852314de4870ba9b3626051ad0c6/roles/21156560eb6b436bb567a40b4976f3f4` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:49.242853 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e350cb-247e-415b-a558-5228cf7cb378 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e62fdabd05bc4fdc96e74c1d09d3ae11, user_id=82ad852314de4870ba9b3626051ad0c6, role_id=21156560eb6b436bb567a40b4976f3f4)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:49.247355 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e350cb-247e-415b-a558-5228cf7cb378 admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:49.266101 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1006/2040] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 10:48:49 2022] PUT /identity/v3/projects/e62fdabd05bc4fdc96e74c1d09d3ae11/users/82ad852314de4870ba9b3626051ad0c6/roles/21156560eb6b436bb567a40b4976f3f4 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:48:49.268561 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e350cb-247e-415b-a558-5228cf7cb378 admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:49.368681 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1035/2041] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:48:49 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:48:49.370537 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6904788c-85cd-4833-9331-c1cdf8dbb92e None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:49.371390 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6904788c-85cd-4833-9331-c1cdf8dbb92e None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:49.371673 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6904788c-85cd-4833-9331-c1cdf8dbb92e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:49.402771 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e350cb-247e-415b-a558-5228cf7cb378 admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:49.419873 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e432630-fec8-4401-bc65-6308652934af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:49.421023 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e432630-fec8-4401-bc65-6308652934af admin admin] REQUEST_METHOD: `PUT` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:49.421316 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e432630-fec8-4401-bc65-6308652934af admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:49.421771 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e432630-fec8-4401-bc65-6308652934af admin admin] PATH_INFO: `/v3/projects/21c10e481b60469da940254d0ea10fb4/users/ccd048bb2e0847ada5f6aed923672dea/roles/21156560eb6b436bb567a40b4976f3f4` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:49.426329 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e432630-fec8-4401-bc65-6308652934af admin admin] RBAC: Authorizing `identity:create_grant(project_id=21c10e481b60469da940254d0ea10fb4, user_id=ccd048bb2e0847ada5f6aed923672dea, role_id=21156560eb6b436bb567a40b4976f3f4)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:49.429430 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e432630-fec8-4401-bc65-6308652934af admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:49.438304 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-6904788c-85cd-4833-9331-c1cdf8dbb92e None None] MFA Rules not processed for user `82ad852314de4870ba9b3626051ad0c6`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:48:49.439829 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-6904788c-85cd-4833-9331-c1cdf8dbb92e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:49.466633 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1007/2042] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 10:48:49 2022] PUT /identity/v3/projects/21c10e481b60469da940254d0ea10fb4/users/ccd048bb2e0847ada5f6aed923672dea/roles/21156560eb6b436bb567a40b4976f3f4 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:48:49.468709 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e432630-fec8-4401-bc65-6308652934af admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:49.478168 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:48:49.589282 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1036/2043] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:48:49 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:48:49.596454 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b9374d-068f-4c64-87bf-f14756f1afb3 None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:49.596759 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b9374d-068f-4c64-87bf-f14756f1afb3 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:49.597044 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b9374d-068f-4c64-87bf-f14756f1afb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:49.608267 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:48:49.629728 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-52b9374d-068f-4c64-87bf-f14756f1afb3 None None] MFA Rules not processed for user `ccd048bb2e0847ada5f6aed923672dea`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:48:49.631226 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-52b9374d-068f-4c64-87bf-f14756f1afb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:49.633693 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e432630-fec8-4401-bc65-6308652934af admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:49.640614 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10da4b2-58d2-4721-9e11-dba8795a80f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:49.641206 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10da4b2-58d2-4721-9e11-dba8795a80f5 admin admin] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:49.641509 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10da4b2-58d2-4721-9e11-dba8795a80f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:49.642060 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10da4b2-58d2-4721-9e11-dba8795a80f5 admin admin] PATH_INFO: `/v3/projects` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:49.642635 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10da4b2-58d2-4721-9e11-dba8795a80f5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:49.644975 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10da4b2-58d2-4721-9e11-dba8795a80f5 admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:49.686931 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1008/2044] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Jul 27 10:48:49 2022] POST /identity/v3/projects => generated 387 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 10:48:49.687934 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10da4b2-58d2-4721-9e11-dba8795a80f5 admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:49.736647 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1037/2045] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:48:49 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:48:49.739606 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b9374d-068f-4c64-87bf-f14756f1afb3 None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:49.764630 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b9374d-068f-4c64-87bf-f14756f1afb3 None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:49.773280 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10da4b2-58d2-4721-9e11-dba8795a80f5 admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:49.783439 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770a0ff2-d008-46c4-a8a6-d9b86ab10b74 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '864d35f268cf45148756109759bb2743', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:49.784098 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770a0ff2-d008-46c4-a8a6-d9b86ab10b74 service cinder] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:49.784385 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770a0ff2-d008-46c4-a8a6-d9b86ab10b74 service cinder] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:49.784667 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770a0ff2-d008-46c4-a8a6-d9b86ab10b74 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:49.857205 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2020aa6a-566f-485f-853f-2c6bcffaf0d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:49.857844 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2020aa6a-566f-485f-853f-2c6bcffaf0d7 admin admin] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:49.858123 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2020aa6a-566f-485f-853f-2c6bcffaf0d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:49.858399 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2020aa6a-566f-485f-853f-2c6bcffaf0d7 admin admin] PATH_INFO: `/v3/users` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:49.859106 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2020aa6a-566f-485f-853f-2c6bcffaf0d7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:49.861021 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2020aa6a-566f-485f-853f-2c6bcffaf0d7 admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:49.875679 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770a0ff2-d008-46c4-a8a6-d9b86ab10b74 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:49.877441 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770a0ff2-d008-46c4-a8a6-d9b86ab10b74 service cinder] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:49.887852 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1038/2046] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Jul 27 10:48:49 2022] POST /identity/v3/users => generated 294 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 10:48:49.887852 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2020aa6a-566f-485f-853f-2c6bcffaf0d7 admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:49.888411 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1009/2047] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:48:49 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 10:48:49.889143 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770a0ff2-d008-46c4-a8a6-d9b86ab10b74 service cinder] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:49.899230 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770a0ff2-d008-46c4-a8a6-d9b86ab10b74 service cinder] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:49.904747 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7c4961-fd6e-4d38-8e5b-50f6a50bdabb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:49.906034 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7c4961-fd6e-4d38-8e5b-50f6a50bdabb admin admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:49.906331 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7c4961-fd6e-4d38-8e5b-50f6a50bdabb admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:49.906757 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7c4961-fd6e-4d38-8e5b-50f6a50bdabb admin admin] PATH_INFO: `/v3/roles` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:49.907983 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7c4961-fd6e-4d38-8e5b-50f6a50bdabb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:49.910162 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7c4961-fd6e-4d38-8e5b-50f6a50bdabb admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:49.911381 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2020aa6a-566f-485f-853f-2c6bcffaf0d7 admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:49.915653 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76e5cfd-646c-44e8-84ad-d8a8c7537c45 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '864d35f268cf45148756109759bb2743', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:49.916513 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76e5cfd-646c-44e8-84ad-d8a8c7537c45 service cinder] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:49.917024 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76e5cfd-646c-44e8-84ad-d8a8c7537c45 service cinder] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:49.917501 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76e5cfd-646c-44e8-84ad-d8a8c7537c45 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:49.920246 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-c76e5cfd-646c-44e8-84ad-d8a8c7537c45 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:49.924356 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1010/2048] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 10:48:49 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 10:48:49.928113 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7c4961-fd6e-4d38-8e5b-50f6a50bdabb admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:49.948421 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:48:49.953330 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7c4961-fd6e-4d38-8e5b-50f6a50bdabb admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:49.960532 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc512d5-5430-4791-a626-649b2fb9dd9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:49.961394 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc512d5-5430-4791-a626-649b2fb9dd9e admin admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:49.961746 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc512d5-5430-4791-a626-649b2fb9dd9e admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:49.962057 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc512d5-5430-4791-a626-649b2fb9dd9e admin admin] PATH_INFO: `/v3/roles` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:49.962870 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc512d5-5430-4791-a626-649b2fb9dd9e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:49.964568 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc512d5-5430-4791-a626-649b2fb9dd9e admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:49.975970 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1011/2049] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 10:48:49 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 10:48:49.982137 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc512d5-5430-4791-a626-649b2fb9dd9e admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:49.995094 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc512d5-5430-4791-a626-649b2fb9dd9e admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:50.000590 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14987cca-9298-4cfb-98a6-82f8a5da3431 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:50.001286 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14987cca-9298-4cfb-98a6-82f8a5da3431 admin admin] REQUEST_METHOD: `PUT` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:50.001828 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14987cca-9298-4cfb-98a6-82f8a5da3431 admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:50.002101 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14987cca-9298-4cfb-98a6-82f8a5da3431 admin admin] PATH_INFO: `/v3/projects/3bac58bc4270449eae3fa7b995fcf34e/users/a874864af3b2496b8dbf359b39f51459/roles/4c9fc0b3baed441da2126be41cc6b04a` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:50.013373 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76e5cfd-646c-44e8-84ad-d8a8c7537c45 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:50.015096 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76e5cfd-646c-44e8-84ad-d8a8c7537c45 service cinder] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:50.020817 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14987cca-9298-4cfb-98a6-82f8a5da3431 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3bac58bc4270449eae3fa7b995fcf34e, user_id=a874864af3b2496b8dbf359b39f51459, role_id=4c9fc0b3baed441da2126be41cc6b04a)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:50.024878 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14987cca-9298-4cfb-98a6-82f8a5da3431 admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:50.034877 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1039/2050] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:48:49 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 10:48:50.042322 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1012/2051] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 10:48:49 2022] PUT /identity/v3/projects/3bac58bc4270449eae3fa7b995fcf34e/users/a874864af3b2496b8dbf359b39f51459/roles/4c9fc0b3baed441da2126be41cc6b04a => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:48:50.054731 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e415940d-d776-4ef0-8f78-16afd88f6572 None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:50.055049 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e415940d-d776-4ef0-8f78-16afd88f6572 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:50.055365 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e415940d-d776-4ef0-8f78-16afd88f6572 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:50.075285 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14987cca-9298-4cfb-98a6-82f8a5da3431 admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:50.107267 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-e415940d-d776-4ef0-8f78-16afd88f6572 None None] MFA Rules not processed for user `a874864af3b2496b8dbf359b39f51459`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:48:50.108845 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-e415940d-d776-4ef0-8f78-16afd88f6572 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:50.125670 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:48:50.164296 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14987cca-9298-4cfb-98a6-82f8a5da3431 admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:50.172871 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26887af9-8bf8-4623-8d56-7b207782c788 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '864d35f268cf45148756109759bb2743', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:50.174340 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26887af9-8bf8-4623-8d56-7b207782c788 service cinder] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:50.175602 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26887af9-8bf8-4623-8d56-7b207782c788 service cinder] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:50.175894 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26887af9-8bf8-4623-8d56-7b207782c788 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:50.224369 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1040/2052] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:48:50 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:48:50.229061 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e415940d-d776-4ef0-8f78-16afd88f6572 None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:50.264636 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26887af9-8bf8-4623-8d56-7b207782c788 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:50.266164 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26887af9-8bf8-4623-8d56-7b207782c788 service cinder] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:50.278400 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1013/2053] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:48:50 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 10:48:50.344526 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e415940d-d776-4ef0-8f78-16afd88f6572 None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:50.356496 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8a0ba1-1cf8-4fa3-b5a7-8f6107550bad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:50.357238 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8a0ba1-1cf8-4fa3-b5a7-8f6107550bad admin admin] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:50.360387 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8a0ba1-1cf8-4fa3-b5a7-8f6107550bad admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:50.360953 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8a0ba1-1cf8-4fa3-b5a7-8f6107550bad admin admin] PATH_INFO: `/v3/projects` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:50.365109 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8a0ba1-1cf8-4fa3-b5a7-8f6107550bad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:50.367102 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8a0ba1-1cf8-4fa3-b5a7-8f6107550bad admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:50.425618 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1041/2054] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Jul 27 10:48:50 2022] POST /identity/v3/projects => generated 385 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 10:48:50.429419 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26887af9-8bf8-4623-8d56-7b207782c788 service cinder] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:50.443743 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26887af9-8bf8-4623-8d56-7b207782c788 service cinder] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:50.581203 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da462e7a-6951-4c12-b4b9-707e6019d4fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:50.581985 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da462e7a-6951-4c12-b4b9-707e6019d4fa admin admin] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:50.582851 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da462e7a-6951-4c12-b4b9-707e6019d4fa admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:50.583210 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da462e7a-6951-4c12-b4b9-707e6019d4fa admin admin] PATH_INFO: `/v3/users` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:50.584976 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da462e7a-6951-4c12-b4b9-707e6019d4fa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:50.588218 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da462e7a-6951-4c12-b4b9-707e6019d4fa admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:50.624760 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1014/2055] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Jul 27 10:48:50 2022] POST /identity/v3/users => generated 299 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 10:48:50.628900 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8a0ba1-1cf8-4fa3-b5a7-8f6107550bad admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:50.644543 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8a0ba1-1cf8-4fa3-b5a7-8f6107550bad admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:50.652646 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f1a188-4d4f-4d68-89f4-4eb3d89d70fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:50.653346 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f1a188-4d4f-4d68-89f4-4eb3d89d70fc admin admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:50.653623 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f1a188-4d4f-4d68-89f4-4eb3d89d70fc admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:50.653904 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f1a188-4d4f-4d68-89f4-4eb3d89d70fc admin admin] PATH_INFO: `/v3/roles` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:50.655061 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f1a188-4d4f-4d68-89f4-4eb3d89d70fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:50.657461 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f1a188-4d4f-4d68-89f4-4eb3d89d70fc admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:50.677816 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1042/2056] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 10:48:50 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 10:48:50.678184 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da462e7a-6951-4c12-b4b9-707e6019d4fa admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:50.690874 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da462e7a-6951-4c12-b4b9-707e6019d4fa admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:50.697313 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccc444b-7fb9-4727-ae48-6c6f713e96de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:50.698117 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccc444b-7fb9-4727-ae48-6c6f713e96de admin admin] REQUEST_METHOD: `PUT` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:50.698477 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccc444b-7fb9-4727-ae48-6c6f713e96de admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:50.699392 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccc444b-7fb9-4727-ae48-6c6f713e96de admin admin] PATH_INFO: `/v3/projects/c5cdef5e9520477b820bb8df4bcbd720/users/d58d04a3e9d84ef5b621cd7e30d1d72b/roles/21156560eb6b436bb567a40b4976f3f4` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:50.720785 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ccc444b-7fb9-4727-ae48-6c6f713e96de admin admin] RBAC: Authorizing `identity:create_grant(project_id=c5cdef5e9520477b820bb8df4bcbd720, user_id=d58d04a3e9d84ef5b621cd7e30d1d72b, role_id=21156560eb6b436bb567a40b4976f3f4)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:50.723828 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ccc444b-7fb9-4727-ae48-6c6f713e96de admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:50.741433 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1015/2057] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 10:48:50 2022] PUT /identity/v3/projects/c5cdef5e9520477b820bb8df4bcbd720/users/d58d04a3e9d84ef5b621cd7e30d1d72b/roles/21156560eb6b436bb567a40b4976f3f4 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:48:50.745340 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f1a188-4d4f-4d68-89f4-4eb3d89d70fc admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:50.759614 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f1a188-4d4f-4d68-89f4-4eb3d89d70fc admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:50.820939 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:48:50.896877 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1480afd6-2a4b-4474-b742-f238670dd4ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:50.898748 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1480afd6-2a4b-4474-b742-f238670dd4ca admin admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:50.899042 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1480afd6-2a4b-4474-b742-f238670dd4ca admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:50.899324 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1480afd6-2a4b-4474-b742-f238670dd4ca admin admin] PATH_INFO: `/v3/roles` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:50.900046 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1480afd6-2a4b-4474-b742-f238670dd4ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:50.902048 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1480afd6-2a4b-4474-b742-f238670dd4ca admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:50.917788 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1043/2058] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 10:48:50 2022] GET /identity/v3/roles => generated 1420 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 10:48:50.919208 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccc444b-7fb9-4727-ae48-6c6f713e96de admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:50.931703 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccc444b-7fb9-4727-ae48-6c6f713e96de admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:50.946712 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3daf4ef9-3df8-4574-a341-d54cf5c209ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:50.947421 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daf4ef9-3df8-4574-a341-d54cf5c209ed admin admin] REQUEST_METHOD: `PUT` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:50.947731 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daf4ef9-3df8-4574-a341-d54cf5c209ed admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:50.948858 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daf4ef9-3df8-4574-a341-d54cf5c209ed admin admin] PATH_INFO: `/v3/projects/c5cdef5e9520477b820bb8df4bcbd720/users/d58d04a3e9d84ef5b621cd7e30d1d72b/roles/21156560eb6b436bb567a40b4976f3f4` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:50.955369 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3daf4ef9-3df8-4574-a341-d54cf5c209ed admin admin] RBAC: Authorizing `identity:create_grant(project_id=c5cdef5e9520477b820bb8df4bcbd720, user_id=d58d04a3e9d84ef5b621cd7e30d1d72b, role_id=21156560eb6b436bb567a40b4976f3f4)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:50.965513 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3daf4ef9-3df8-4574-a341-d54cf5c209ed admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:50.989833 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1016/2059] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 10:48:50 2022] PUT /identity/v3/projects/c5cdef5e9520477b820bb8df4bcbd720/users/d58d04a3e9d84ef5b621cd7e30d1d72b/roles/21156560eb6b436bb567a40b4976f3f4 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:48:51.000498 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7603f96-63e9-4be4-973d-fdc87b29cd86 None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:51.000803 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7603f96-63e9-4be4-973d-fdc87b29cd86 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:51.001079 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7603f96-63e9-4be4-973d-fdc87b29cd86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:51.063654 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-c7603f96-63e9-4be4-973d-fdc87b29cd86 None None] MFA Rules not processed for user `d58d04a3e9d84ef5b621cd7e30d1d72b`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:48:51.065104 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-c7603f96-63e9-4be4-973d-fdc87b29cd86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:51.131220 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3daf4ef9-3df8-4574-a341-d54cf5c209ed admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:51.209916 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1044/2060] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:48:50 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:48:51.219418 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7603f96-63e9-4be4-973d-fdc87b29cd86 None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:51.225865 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:48:51.256220 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3daf4ef9-3df8-4574-a341-d54cf5c209ed admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:51.269657 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d77fcd0-570b-4bd7-8cb7-521c59c41acb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:51.272162 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d77fcd0-570b-4bd7-8cb7-521c59c41acb admin admin] REQUEST_METHOD: `DELETE` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:51.272528 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d77fcd0-570b-4bd7-8cb7-521c59c41acb admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:51.273126 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d77fcd0-570b-4bd7-8cb7-521c59c41acb admin admin] PATH_INFO: `/v3/users/cc81658ff1bc48c1b5b4cfc1d19ee9fe` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:51.274572 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d77fcd0-570b-4bd7-8cb7-521c59c41acb admin admin] RBAC: Authorizing `identity:delete_user(user_id=cc81658ff1bc48c1b5b4cfc1d19ee9fe)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:51.278067 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d77fcd0-570b-4bd7-8cb7-521c59c41acb admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:51.298220 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7603f96-63e9-4be4-973d-fdc87b29cd86 None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:51.301924 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa53f89-4527-49cf-945b-460c3693a3e1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '864d35f268cf45148756109759bb2743', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:51.302561 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa53f89-4527-49cf-945b-460c3693a3e1 service cinder] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:51.303402 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa53f89-4527-49cf-945b-460c3693a3e1 service cinder] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:51.303832 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa53f89-4527-49cf-945b-460c3693a3e1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:51.316184 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa53f89-4527-49cf-945b-460c3693a3e1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:51.318004 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa53f89-4527-49cf-945b-460c3693a3e1 service cinder] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:51.331186 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1045/2061] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:48:51 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 10:48:51.353532 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-1d77fcd0-570b-4bd7-8cb7-521c59c41acb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cc81658ff1bc48c1b5b4cfc1d19ee9fe'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:51.369150 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-1d77fcd0-570b-4bd7-8cb7-521c59c41acb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cc81658ff1bc48c1b5b4cfc1d19ee9fe'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:51.381219 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-1d77fcd0-570b-4bd7-8cb7-521c59c41acb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cc81658ff1bc48c1b5b4cfc1d19ee9fe'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:51.382575 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-1d77fcd0-570b-4bd7-8cb7-521c59c41acb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cc81658ff1bc48c1b5b4cfc1d19ee9fe'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:51.384057 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-1d77fcd0-570b-4bd7-8cb7-521c59c41acb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cc81658ff1bc48c1b5b4cfc1d19ee9fe'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:51.411828 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1017/2062] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Jul 27 10:48:51 2022] DELETE /identity/v3/users/cc81658ff1bc48c1b5b4cfc1d19ee9fe => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 10:48:51.416885 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c85e5fa-4024-486e-8220-244f2b3a15a2 None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:51.417234 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c85e5fa-4024-486e-8220-244f2b3a15a2 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:51.418659 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c85e5fa-4024-486e-8220-244f2b3a15a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:51.466536 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-0c85e5fa-4024-486e-8220-244f2b3a15a2 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:48:51.468068 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-0c85e5fa-4024-486e-8220-244f2b3a15a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:51.537301 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:48:51.587375 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1046/2063] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:48:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:48:51.596448 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d77fcd0-570b-4bd7-8cb7-521c59c41acb admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:51.599153 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-1d77fcd0-570b-4bd7-8cb7-521c59c41acb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:51.749597 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d77fcd0-570b-4bd7-8cb7-521c59c41acb admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:51.754242 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13375723-19e5-498c-b8f0-af2af090145c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7922d0a1a7c4e808d0b2a80dca0da40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:51.754937 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13375723-19e5-498c-b8f0-af2af090145c service neutron] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:51.755217 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13375723-19e5-498c-b8f0-af2af090145c service neutron] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:51.755486 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13375723-19e5-498c-b8f0-af2af090145c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:51.767739 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13375723-19e5-498c-b8f0-af2af090145c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:51.769171 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13375723-19e5-498c-b8f0-af2af090145c service neutron] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:51.798721 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1018/2064] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:48:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 10:48:51.823433 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c85e5fa-4024-486e-8220-244f2b3a15a2 None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:51.825758 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-0c85e5fa-4024-486e-8220-244f2b3a15a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:51.919712 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c85e5fa-4024-486e-8220-244f2b3a15a2 None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:51.925534 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91df1c15-c717-49fa-be64-fed1f08f5ffd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '864d35f268cf45148756109759bb2743', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:51.926338 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91df1c15-c717-49fa-be64-fed1f08f5ffd service cinder] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:51.926682 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91df1c15-c717-49fa-be64-fed1f08f5ffd service cinder] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:51.927131 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91df1c15-c717-49fa-be64-fed1f08f5ffd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:51.929567 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-91df1c15-c717-49fa-be64-fed1f08f5ffd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:52.017061 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91df1c15-c717-49fa-be64-fed1f08f5ffd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:52.018513 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91df1c15-c717-49fa-be64-fed1f08f5ffd service cinder] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:52.029435 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1047/2065] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:48:51 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 10:48:52.262703 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13375723-19e5-498c-b8f0-af2af090145c service neutron] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:52.264172 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-13375723-19e5-498c-b8f0-af2af090145c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:52.270993 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91df1c15-c717-49fa-be64-fed1f08f5ffd service cinder] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:52.272472 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-91df1c15-c717-49fa-be64-fed1f08f5ffd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:52.280928 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13375723-19e5-498c-b8f0-af2af090145c service neutron] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:52.284985 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6105a7-14a5-4494-bf4a-ea8c5f1e4124 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '864d35f268cf45148756109759bb2743', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:52.285868 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6105a7-14a5-4494-bf4a-ea8c5f1e4124 service cinder] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:52.286330 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6105a7-14a5-4494-bf4a-ea8c5f1e4124 service cinder] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:52.286776 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6105a7-14a5-4494-bf4a-ea8c5f1e4124 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:52.288907 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-ab6105a7-14a5-4494-bf4a-ea8c5f1e4124 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:52.294405 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91df1c15-c717-49fa-be64-fed1f08f5ffd service cinder] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:52.300177 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f644a14-dc94-406f-8a8c-ce6b056b087d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:52.300784 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f644a14-dc94-406f-8a8c-ce6b056b087d admin admin] REQUEST_METHOD: `DELETE` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:52.301069 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f644a14-dc94-406f-8a8c-ce6b056b087d admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:52.301346 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f644a14-dc94-406f-8a8c-ce6b056b087d admin admin] PATH_INFO: `/v3/projects/7ca07b06b6a94d30a8e58ec294519ee9` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:52.302520 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f644a14-dc94-406f-8a8c-ce6b056b087d admin admin] RBAC: Authorizing `identity:delete_project(project_id=7ca07b06b6a94d30a8e58ec294519ee9)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:52.304478 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f644a14-dc94-406f-8a8c-ce6b056b087d admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:52.360847 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-1f644a14-dc94-406f-8a8c-ce6b056b087d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7ca07b06b6a94d30a8e58ec294519ee9', 'request_id': 'req-1f644a14-dc94-406f-8a8c-ce6b056b087d'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:52.364647 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6105a7-14a5-4494-bf4a-ea8c5f1e4124 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:52.366172 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6105a7-14a5-4494-bf4a-ea8c5f1e4124 service cinder] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:52.367286 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-1f644a14-dc94-406f-8a8c-ce6b056b087d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7ca07b06b6a94d30a8e58ec294519ee9', 'request_id': 'req-1f644a14-dc94-406f-8a8c-ce6b056b087d'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:52.371091 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:48:52.376831 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-1f644a14-dc94-406f-8a8c-ce6b056b087d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7ca07b06b6a94d30a8e58ec294519ee9', 'request_id': 'req-1f644a14-dc94-406f-8a8c-ce6b056b087d'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:52.378142 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1019/2066] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:48:52 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 10:48:52.392955 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-1f644a14-dc94-406f-8a8c-ce6b056b087d admin admin] The token cache is being invalidate because project 7ca07b06b6a94d30a8e58ec294519ee9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86827) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 10:48:52.393248 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-1f644a14-dc94-406f-8a8c-ce6b056b087d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:52.393934 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-1f644a14-dc94-406f-8a8c-ce6b056b087d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:52.398527 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1048/2067] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Jul 27 10:48:52 2022] DELETE /identity/v3/projects/7ca07b06b6a94d30a8e58ec294519ee9 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:48:52.399685 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6105a7-14a5-4494-bf4a-ea8c5f1e4124 service cinder] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:52.410798 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6105a7-14a5-4494-bf4a-ea8c5f1e4124 service cinder] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:52.469622 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3940b669-ddab-4fce-a2cb-a852b18f55b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:52.470293 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3940b669-ddab-4fce-a2cb-a852b18f55b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:52.470736 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3940b669-ddab-4fce-a2cb-a852b18f55b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:52.471019 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3940b669-ddab-4fce-a2cb-a852b18f55b4 admin admin] PATH_INFO: `/v3/users/82ad852314de4870ba9b3626051ad0c6` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:52.472219 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3940b669-ddab-4fce-a2cb-a852b18f55b4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=82ad852314de4870ba9b3626051ad0c6)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:52.474143 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3940b669-ddab-4fce-a2cb-a852b18f55b4 admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:52.526100 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-3940b669-ddab-4fce-a2cb-a852b18f55b4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '82ad852314de4870ba9b3626051ad0c6'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:52.535266 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-3940b669-ddab-4fce-a2cb-a852b18f55b4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '82ad852314de4870ba9b3626051ad0c6'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:52.543601 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-3940b669-ddab-4fce-a2cb-a852b18f55b4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '82ad852314de4870ba9b3626051ad0c6'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:52.544329 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-3940b669-ddab-4fce-a2cb-a852b18f55b4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '82ad852314de4870ba9b3626051ad0c6'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:52.545017 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-3940b669-ddab-4fce-a2cb-a852b18f55b4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '82ad852314de4870ba9b3626051ad0c6'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:52.563716 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1020/2068] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Jul 27 10:48:52 2022] DELETE /identity/v3/users/82ad852314de4870ba9b3626051ad0c6 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:48:52.858345 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f644a14-dc94-406f-8a8c-ce6b056b087d admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:52.859879 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-1f644a14-dc94-406f-8a8c-ce6b056b087d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:52.873394 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f644a14-dc94-406f-8a8c-ce6b056b087d admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:52.903216 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:48:52.935325 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9cfb84-5c01-4b6e-ab04-3886942dc153 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:52.935976 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9cfb84-5c01-4b6e-ab04-3886942dc153 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:52.936266 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9cfb84-5c01-4b6e-ab04-3886942dc153 admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:52.936546 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9cfb84-5c01-4b6e-ab04-3886942dc153 admin admin] PATH_INFO: `/v3/projects/e62fdabd05bc4fdc96e74c1d09d3ae11` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:52.937710 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9cfb84-5c01-4b6e-ab04-3886942dc153 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e62fdabd05bc4fdc96e74c1d09d3ae11)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:52.939664 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9cfb84-5c01-4b6e-ab04-3886942dc153 admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:52.987631 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-ca9cfb84-5c01-4b6e-ab04-3886942dc153 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e62fdabd05bc4fdc96e74c1d09d3ae11', 'request_id': 'req-ca9cfb84-5c01-4b6e-ab04-3886942dc153'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:52.993658 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-ca9cfb84-5c01-4b6e-ab04-3886942dc153 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e62fdabd05bc4fdc96e74c1d09d3ae11', 'request_id': 'req-ca9cfb84-5c01-4b6e-ab04-3886942dc153'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:53.001230 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-ca9cfb84-5c01-4b6e-ab04-3886942dc153 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e62fdabd05bc4fdc96e74c1d09d3ae11', 'request_id': 'req-ca9cfb84-5c01-4b6e-ab04-3886942dc153'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:53.012480 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-ca9cfb84-5c01-4b6e-ab04-3886942dc153 admin admin] The token cache is being invalidate because project e62fdabd05bc4fdc96e74c1d09d3ae11 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86827) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 10:48:53.012772 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-ca9cfb84-5c01-4b6e-ab04-3886942dc153 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:53.013391 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-ca9cfb84-5c01-4b6e-ab04-3886942dc153 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:53.014943 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1049/2069] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Jul 27 10:48:52 2022] DELETE /identity/v3/projects/e62fdabd05bc4fdc96e74c1d09d3ae11 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:48:53.019806 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c1c202-da16-4877-96be-bbc5a75ab0af None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:53.020098 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c1c202-da16-4877-96be-bbc5a75ab0af None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:53.020382 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c1c202-da16-4877-96be-bbc5a75ab0af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:53.040186 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-e4c1c202-da16-4877-96be-bbc5a75ab0af None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:48:53.041396 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-e4c1c202-da16-4877-96be-bbc5a75ab0af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:53.101425 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1021/2070] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:48:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:48:53.109265 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8093eb5-76cb-4b99-a0d6-6e62636885a2 None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:53.109561 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8093eb5-76cb-4b99-a0d6-6e62636885a2 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:53.109846 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8093eb5-76cb-4b99-a0d6-6e62636885a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:53.129839 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-e8093eb5-76cb-4b99-a0d6-6e62636885a2 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:48:53.131073 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-e8093eb5-76cb-4b99-a0d6-6e62636885a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:53.139761 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1050/2071] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:48:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:48:53.144073 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c1c202-da16-4877-96be-bbc5a75ab0af None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:53.154163 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c1c202-da16-4877-96be-bbc5a75ab0af None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:53.158702 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579a37cd-0940-435f-b3b9-f8f30e6c57d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:53.159311 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579a37cd-0940-435f-b3b9-f8f30e6c57d1 admin admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:53.159590 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579a37cd-0940-435f-b3b9-f8f30e6c57d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:53.159860 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579a37cd-0940-435f-b3b9-f8f30e6c57d1 admin admin] PATH_INFO: `/v3/domains` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:53.160575 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579a37cd-0940-435f-b3b9-f8f30e6c57d1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86828) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:48:53.160885 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579a37cd-0940-435f-b3b9-f8f30e6c57d1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:53.162464 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579a37cd-0940-435f-b3b9-f8f30e6c57d1 admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:53.174608 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1022/2072] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Jul 27 10:48:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:48:53.179047 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8093eb5-76cb-4b99-a0d6-6e62636885a2 None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:53.188653 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8093eb5-76cb-4b99-a0d6-6e62636885a2 None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:53.193565 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e881c6b-1668-4efc-a2d7-71b539e7cc2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:53.198442 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e881c6b-1668-4efc-a2d7-71b539e7cc2c admin admin] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:53.198806 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e881c6b-1668-4efc-a2d7-71b539e7cc2c admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:53.199091 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e881c6b-1668-4efc-a2d7-71b539e7cc2c admin admin] PATH_INFO: `/v3/projects` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:53.199821 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e881c6b-1668-4efc-a2d7-71b539e7cc2c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:53.201811 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e881c6b-1668-4efc-a2d7-71b539e7cc2c admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:53.242541 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1051/2073] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Jul 27 10:48:53 2022] POST /identity/v3/projects => generated 361 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 10:48:53.243623 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579a37cd-0940-435f-b3b9-f8f30e6c57d1 admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:53.251223 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:48:53.255695 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579a37cd-0940-435f-b3b9-f8f30e6c57d1 admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:53.317073 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f8a0b3-6b7f-4f34-91e4-84b9b204fc4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:53.317719 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f8a0b3-6b7f-4f34-91e4-84b9b204fc4d admin admin] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:53.317969 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f8a0b3-6b7f-4f34-91e4-84b9b204fc4d admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:53.318243 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f8a0b3-6b7f-4f34-91e4-84b9b204fc4d admin admin] PATH_INFO: `/v3/users` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:53.318940 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f8a0b3-6b7f-4f34-91e4-84b9b204fc4d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:53.320880 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f8a0b3-6b7f-4f34-91e4-84b9b204fc4d admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:53.343471 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1023/2074] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Jul 27 10:48:53 2022] POST /identity/v3/users => generated 281 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 10:48:53.347740 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e881c6b-1668-4efc-a2d7-71b539e7cc2c admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:53.358902 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e881c6b-1668-4efc-a2d7-71b539e7cc2c admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:53.363427 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47fa7ca-19d2-4607-8d72-cbc17a6f074a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:53.364032 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47fa7ca-19d2-4607-8d72-cbc17a6f074a admin admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:53.364312 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47fa7ca-19d2-4607-8d72-cbc17a6f074a admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:53.364583 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47fa7ca-19d2-4607-8d72-cbc17a6f074a admin admin] PATH_INFO: `/v3/roles` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:53.365269 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47fa7ca-19d2-4607-8d72-cbc17a6f074a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:53.366886 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47fa7ca-19d2-4607-8d72-cbc17a6f074a admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:53.378127 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1052/2075] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 10:48:53 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 10:48:53.381932 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f8a0b3-6b7f-4f34-91e4-84b9b204fc4d admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:53.391732 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f8a0b3-6b7f-4f34-91e4-84b9b204fc4d admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:53.396351 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3036d1-2571-48c2-a93c-b123912dcf58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:53.396966 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3036d1-2571-48c2-a93c-b123912dcf58 admin admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:53.397250 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3036d1-2571-48c2-a93c-b123912dcf58 admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:53.397529 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3036d1-2571-48c2-a93c-b123912dcf58 admin admin] PATH_INFO: `/v3/roles` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:53.398214 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3036d1-2571-48c2-a93c-b123912dcf58 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:53.399844 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3036d1-2571-48c2-a93c-b123912dcf58 admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:53.410850 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1024/2076] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 10:48:53 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 10:48:53.414809 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47fa7ca-19d2-4607-8d72-cbc17a6f074a admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:53.425243 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47fa7ca-19d2-4607-8d72-cbc17a6f074a admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:53.429855 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8304da08-5a90-4a0f-9051-3f9dadc5c826 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:53.430513 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8304da08-5a90-4a0f-9051-3f9dadc5c826 admin admin] REQUEST_METHOD: `PUT` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:53.430825 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8304da08-5a90-4a0f-9051-3f9dadc5c826 admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:53.431132 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8304da08-5a90-4a0f-9051-3f9dadc5c826 admin admin] PATH_INFO: `/v3/projects/5c21caa4be5e4ea3b7b846fe498d648b/users/29cd27e547e94fc0b8e2d2572f5838f2/roles/4c9fc0b3baed441da2126be41cc6b04a` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:53.445315 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8304da08-5a90-4a0f-9051-3f9dadc5c826 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5c21caa4be5e4ea3b7b846fe498d648b, user_id=29cd27e547e94fc0b8e2d2572f5838f2, role_id=4c9fc0b3baed441da2126be41cc6b04a)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:53.448015 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8304da08-5a90-4a0f-9051-3f9dadc5c826 admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:53.461358 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1053/2077] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 10:48:53 2022] PUT /identity/v3/projects/5c21caa4be5e4ea3b7b846fe498d648b/users/29cd27e547e94fc0b8e2d2572f5838f2/roles/4c9fc0b3baed441da2126be41cc6b04a => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:48:53.472844 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56430d93-39fd-408a-b174-b5dd449fec3b None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:53.473138 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56430d93-39fd-408a-b174-b5dd449fec3b None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:53.473446 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56430d93-39fd-408a-b174-b5dd449fec3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:53.503067 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-56430d93-39fd-408a-b174-b5dd449fec3b None None] MFA Rules not processed for user `29cd27e547e94fc0b8e2d2572f5838f2`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:48:53.504301 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-56430d93-39fd-408a-b174-b5dd449fec3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:53.570212 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1025/2078] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:48:53 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 10:48:53.602246 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8304da08-5a90-4a0f-9051-3f9dadc5c826 admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:53.603949 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-8304da08-5a90-4a0f-9051-3f9dadc5c826 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:53.661264 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8304da08-5a90-4a0f-9051-3f9dadc5c826 admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:53.665104 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5365c5e8-a828-4c43-94e2-44b1543c4add service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '864d35f268cf45148756109759bb2743', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:53.665723 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5365c5e8-a828-4c43-94e2-44b1543c4add service cinder] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:53.666011 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5365c5e8-a828-4c43-94e2-44b1543c4add service cinder] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:53.666291 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5365c5e8-a828-4c43-94e2-44b1543c4add service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:53.674922 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5365c5e8-a828-4c43-94e2-44b1543c4add service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:53.676355 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5365c5e8-a828-4c43-94e2-44b1543c4add service cinder] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:53.679875 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:48:53.687039 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1054/2079] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:48:53 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 10:48:53.703674 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56430d93-39fd-408a-b174-b5dd449fec3b None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:53.769670 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56430d93-39fd-408a-b174-b5dd449fec3b None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:53.774689 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e5581c-d657-4147-bcb5-fa7fe019a4b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:53.775394 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e5581c-d657-4147-bcb5-fa7fe019a4b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:53.775640 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e5581c-d657-4147-bcb5-fa7fe019a4b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:53.775920 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e5581c-d657-4147-bcb5-fa7fe019a4b9 admin admin] PATH_INFO: `/v3/users/29cd27e547e94fc0b8e2d2572f5838f2` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:53.777775 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e5581c-d657-4147-bcb5-fa7fe019a4b9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=29cd27e547e94fc0b8e2d2572f5838f2)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:53.779992 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e5581c-d657-4147-bcb5-fa7fe019a4b9 admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:53.794289 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:48:53.828712 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-e5e5581c-d657-4147-bcb5-fa7fe019a4b9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '29cd27e547e94fc0b8e2d2572f5838f2'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:53.835551 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-e5e5581c-d657-4147-bcb5-fa7fe019a4b9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '29cd27e547e94fc0b8e2d2572f5838f2'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:53.843962 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-e5e5581c-d657-4147-bcb5-fa7fe019a4b9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '29cd27e547e94fc0b8e2d2572f5838f2'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:53.844582 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-e5e5581c-d657-4147-bcb5-fa7fe019a4b9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '29cd27e547e94fc0b8e2d2572f5838f2'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:53.845248 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-e5e5581c-d657-4147-bcb5-fa7fe019a4b9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '29cd27e547e94fc0b8e2d2572f5838f2'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:53.863069 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1026/2080] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Jul 27 10:48:53 2022] DELETE /identity/v3/users/29cd27e547e94fc0b8e2d2572f5838f2 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:48:53.867566 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0866ed07-8484-4b6d-b94b-c53de65359a0 None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:53.868072 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0866ed07-8484-4b6d-b94b-c53de65359a0 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:53.868172 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0866ed07-8484-4b6d-b94b-c53de65359a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:53.889342 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-0866ed07-8484-4b6d-b94b-c53de65359a0 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:48:53.890559 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-0866ed07-8484-4b6d-b94b-c53de65359a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:53.954722 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1055/2081] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:48:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:48:53.964306 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e5581c-d657-4147-bcb5-fa7fe019a4b9 admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:53.965737 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-e5e5581c-d657-4147-bcb5-fa7fe019a4b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:54.022609 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e5581c-d657-4147-bcb5-fa7fe019a4b9 admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:54.026228 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e3282a-847c-408a-97de-14671dbf72f5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7922d0a1a7c4e808d0b2a80dca0da40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:54.026906 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e3282a-847c-408a-97de-14671dbf72f5 service neutron] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:54.027193 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e3282a-847c-408a-97de-14671dbf72f5 service neutron] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:54.027468 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e3282a-847c-408a-97de-14671dbf72f5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:54.036331 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e3282a-847c-408a-97de-14671dbf72f5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:54.037754 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e3282a-847c-408a-97de-14671dbf72f5 service neutron] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:54.047173 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1027/2082] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:48:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 10:48:54.345142 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0866ed07-8484-4b6d-b94b-c53de65359a0 None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:54.346692 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-0866ed07-8484-4b6d-b94b-c53de65359a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:54.368517 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0866ed07-8484-4b6d-b94b-c53de65359a0 None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:54.377228 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f65736-a6dd-4992-a50b-c659a7efc9ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:54.383318 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f65736-a6dd-4992-a50b-c659a7efc9ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:54.383318 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f65736-a6dd-4992-a50b-c659a7efc9ae admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:54.383318 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f65736-a6dd-4992-a50b-c659a7efc9ae admin admin] PATH_INFO: `/v3/projects/5c21caa4be5e4ea3b7b846fe498d648b` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:54.383318 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f65736-a6dd-4992-a50b-c659a7efc9ae admin admin] RBAC: Authorizing `identity:delete_project(project_id=5c21caa4be5e4ea3b7b846fe498d648b)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:54.395966 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f65736-a6dd-4992-a50b-c659a7efc9ae admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:54.466106 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-a6f65736-a6dd-4992-a50b-c659a7efc9ae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5c21caa4be5e4ea3b7b846fe498d648b', 'request_id': 'req-a6f65736-a6dd-4992-a50b-c659a7efc9ae'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:54.474798 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-a6f65736-a6dd-4992-a50b-c659a7efc9ae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5c21caa4be5e4ea3b7b846fe498d648b', 'request_id': 'req-a6f65736-a6dd-4992-a50b-c659a7efc9ae'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:54.484888 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-a6f65736-a6dd-4992-a50b-c659a7efc9ae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5c21caa4be5e4ea3b7b846fe498d648b', 'request_id': 'req-a6f65736-a6dd-4992-a50b-c659a7efc9ae'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:54.500638 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-a6f65736-a6dd-4992-a50b-c659a7efc9ae admin admin] The token cache is being invalidate because project 5c21caa4be5e4ea3b7b846fe498d648b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86827) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 10:48:54.500971 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-a6f65736-a6dd-4992-a50b-c659a7efc9ae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:54.501907 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-a6f65736-a6dd-4992-a50b-c659a7efc9ae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:48:54.506915 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1056/2083] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Jul 27 10:48:54 2022] DELETE /identity/v3/projects/5c21caa4be5e4ea3b7b846fe498d648b => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:48:54.513315 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5916eadd-b2fc-45e9-9ea4-68cb6e089737 None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:54.514160 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5916eadd-b2fc-45e9-9ea4-68cb6e089737 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:54.515145 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5916eadd-b2fc-45e9-9ea4-68cb6e089737 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:54.536676 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-5916eadd-b2fc-45e9-9ea4-68cb6e089737 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:48:54.537886 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-5916eadd-b2fc-45e9-9ea4-68cb6e089737 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:54.545132 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f65736-a6dd-4992-a50b-c659a7efc9ae admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:54.546520 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-a6f65736-a6dd-4992-a50b-c659a7efc9ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:54.611989 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1028/2084] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:48:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:48:54.613698 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f65736-a6dd-4992-a50b-c659a7efc9ae admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:54.619553 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8569804-6150-4911-9907-e3bc67a1d2fd None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:54.619924 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8569804-6150-4911-9907-e3bc67a1d2fd None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:54.620209 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8569804-6150-4911-9907-e3bc67a1d2fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:54.622350 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7a8dee-f48e-469f-ac22-579a6517e57a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '864d35f268cf45148756109759bb2743', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:54.623274 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7a8dee-f48e-469f-ac22-579a6517e57a service cinder] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:54.623642 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7a8dee-f48e-469f-ac22-579a6517e57a service cinder] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:54.623993 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7a8dee-f48e-469f-ac22-579a6517e57a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:54.626039 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-ca7a8dee-f48e-469f-ac22-579a6517e57a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:54.654627 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-c8569804-6150-4911-9907-e3bc67a1d2fd None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:48:54.654627 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-c8569804-6150-4911-9907-e3bc67a1d2fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:54.656972 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1029/2085] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:48:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 10:48:54.660800 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8569804-6150-4911-9907-e3bc67a1d2fd None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:54.670486 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8569804-6150-4911-9907-e3bc67a1d2fd None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:54.675388 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ca6556-a9db-4a85-be7d-ed1bfbed5348 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:54.676012 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ca6556-a9db-4a85-be7d-ed1bfbed5348 admin admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:54.676295 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ca6556-a9db-4a85-be7d-ed1bfbed5348 admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:54.676568 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ca6556-a9db-4a85-be7d-ed1bfbed5348 admin admin] PATH_INFO: `/v3/domains` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:54.677267 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ca6556-a9db-4a85-be7d-ed1bfbed5348 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86828) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:48:54.677573 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ca6556-a9db-4a85-be7d-ed1bfbed5348 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:54.679214 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ca6556-a9db-4a85-be7d-ed1bfbed5348 admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:54.680450 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:48:54.693112 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1030/2086] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Jul 27 10:48:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 10:48:54.697951 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ca6556-a9db-4a85-be7d-ed1bfbed5348 admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:54.710043 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ca6556-a9db-4a85-be7d-ed1bfbed5348 admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:54.715237 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff24836-0c77-45c0-8d22-8c97d378f99f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:54.715863 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff24836-0c77-45c0-8d22-8c97d378f99f admin admin] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:54.716148 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff24836-0c77-45c0-8d22-8c97d378f99f admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:54.716427 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff24836-0c77-45c0-8d22-8c97d378f99f admin admin] PATH_INFO: `/v3/projects` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:54.717097 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff24836-0c77-45c0-8d22-8c97d378f99f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:54.718087 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7a8dee-f48e-469f-ac22-579a6517e57a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:54.719037 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff24836-0c77-45c0-8d22-8c97d378f99f admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:54.719593 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7a8dee-f48e-469f-ac22-579a6517e57a service cinder] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:54.731956 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1057/2087] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:48:54 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 10:48:54.734961 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:48:54.764875 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7a8dee-f48e-469f-ac22-579a6517e57a service cinder] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:54.771299 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1031/2088] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Jul 27 10:48:54 2022] POST /identity/v3/projects => generated 385 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 10:48:54.776571 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7a8dee-f48e-469f-ac22-579a6517e57a service cinder] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:54.860709 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4599adf0-f156-4e2c-bd8f-9c37bbaa29ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:54.861907 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4599adf0-f156-4e2c-bd8f-9c37bbaa29ea admin admin] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:54.862376 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4599adf0-f156-4e2c-bd8f-9c37bbaa29ea admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:54.862992 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4599adf0-f156-4e2c-bd8f-9c37bbaa29ea admin admin] PATH_INFO: `/v3/users` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:54.868034 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4599adf0-f156-4e2c-bd8f-9c37bbaa29ea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:54.870234 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4599adf0-f156-4e2c-bd8f-9c37bbaa29ea admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:54.909481 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff24836-0c77-45c0-8d22-8c97d378f99f admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:54.914696 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1058/2089] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Jul 27 10:48:54 2022] POST /identity/v3/users => generated 293 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 10:48:54.922000 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff24836-0c77-45c0-8d22-8c97d378f99f admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:54.940039 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e031749-714c-4711-9a6b-83c1b816e10f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:54.940751 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e031749-714c-4711-9a6b-83c1b816e10f admin admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:54.941043 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e031749-714c-4711-9a6b-83c1b816e10f admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:54.941322 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e031749-714c-4711-9a6b-83c1b816e10f admin admin] PATH_INFO: `/v3/roles` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:54.942087 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e031749-714c-4711-9a6b-83c1b816e10f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:54.943991 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e031749-714c-4711-9a6b-83c1b816e10f admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:54.967960 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1032/2090] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 10:48:54 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 10:48:54.972909 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4599adf0-f156-4e2c-bd8f-9c37bbaa29ea admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:54.985938 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4599adf0-f156-4e2c-bd8f-9c37bbaa29ea admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:54.991163 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e139f910-244e-4842-beb7-e5f39ebdc3e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:54.991897 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e139f910-244e-4842-beb7-e5f39ebdc3e4 admin admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:54.992256 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e139f910-244e-4842-beb7-e5f39ebdc3e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:54.992611 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e139f910-244e-4842-beb7-e5f39ebdc3e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:54.994732 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e139f910-244e-4842-beb7-e5f39ebdc3e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:54.995657 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e139f910-244e-4842-beb7-e5f39ebdc3e4 admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:55.014367 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1059/2091] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 10:48:54 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 10:48:55.016461 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e031749-714c-4711-9a6b-83c1b816e10f admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:55.032988 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e031749-714c-4711-9a6b-83c1b816e10f admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:55.045296 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5efe29-f655-42eb-b2d7-1b020da3ecf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:55.046016 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5efe29-f655-42eb-b2d7-1b020da3ecf0 admin admin] REQUEST_METHOD: `PUT` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:55.048033 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5efe29-f655-42eb-b2d7-1b020da3ecf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:55.048412 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5efe29-f655-42eb-b2d7-1b020da3ecf0 admin admin] PATH_INFO: `/v3/projects/9d9fcadac6784925ae34dba862b9db9e/users/67469ca501314670b39dd957b5f8646e/roles/4c9fc0b3baed441da2126be41cc6b04a` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:55.073654 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5efe29-f655-42eb-b2d7-1b020da3ecf0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d9fcadac6784925ae34dba862b9db9e, user_id=67469ca501314670b39dd957b5f8646e, role_id=4c9fc0b3baed441da2126be41cc6b04a)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:55.077155 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5efe29-f655-42eb-b2d7-1b020da3ecf0 admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:55.094908 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1033/2092] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 10:48:55 2022] PUT /identity/v3/projects/9d9fcadac6784925ae34dba862b9db9e/users/67469ca501314670b39dd957b5f8646e/roles/4c9fc0b3baed441da2126be41cc6b04a => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:48:55.106842 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abbf2e3-9b3b-4503-95bd-b6061b452766 None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:55.107216 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abbf2e3-9b3b-4503-95bd-b6061b452766 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:55.107781 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abbf2e3-9b3b-4503-95bd-b6061b452766 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:55.138320 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-8abbf2e3-9b3b-4503-95bd-b6061b452766 None None] MFA Rules not processed for user `67469ca501314670b39dd957b5f8646e`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:48:55.140827 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-8abbf2e3-9b3b-4503-95bd-b6061b452766 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:55.221245 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1060/2093] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:48:55 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:48:55.226808 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5efe29-f655-42eb-b2d7-1b020da3ecf0 admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:55.334838 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5efe29-f655-42eb-b2d7-1b020da3ecf0 admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:55.342020 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e227b079-81bb-4182-9b93-156447bc12c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:55.342844 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e227b079-81bb-4182-9b93-156447bc12c2 admin admin] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:55.343338 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e227b079-81bb-4182-9b93-156447bc12c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:55.343489 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e227b079-81bb-4182-9b93-156447bc12c2 admin admin] PATH_INFO: `/v3/projects` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:55.344130 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e227b079-81bb-4182-9b93-156447bc12c2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:55.346791 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e227b079-81bb-4182-9b93-156447bc12c2 admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:55.384011 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1034/2094] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Jul 27 10:48:55 2022] POST /identity/v3/projects => generated 385 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 10:48:55.387397 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abbf2e3-9b3b-4503-95bd-b6061b452766 None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:55.399616 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abbf2e3-9b3b-4503-95bd-b6061b452766 None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:55.443614 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:48:55.470149 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c750975-9ade-49b2-8e52-d15a37e3786c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:55.470806 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c750975-9ade-49b2-8e52-d15a37e3786c admin admin] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:55.471088 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c750975-9ade-49b2-8e52-d15a37e3786c admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:55.471359 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c750975-9ade-49b2-8e52-d15a37e3786c admin admin] PATH_INFO: `/v3/users` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:55.472325 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c750975-9ade-49b2-8e52-d15a37e3786c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:55.474361 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c750975-9ade-49b2-8e52-d15a37e3786c admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:55.505293 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1061/2095] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Jul 27 10:48:55 2022] POST /identity/v3/users => generated 293 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 10:48:55.506647 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e227b079-81bb-4182-9b93-156447bc12c2 admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:55.520768 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e227b079-81bb-4182-9b93-156447bc12c2 admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:55.525456 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b5609a-6cef-4c07-97ab-8b28441d4c1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:55.526072 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b5609a-6cef-4c07-97ab-8b28441d4c1f admin admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:55.526355 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b5609a-6cef-4c07-97ab-8b28441d4c1f admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:55.526660 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b5609a-6cef-4c07-97ab-8b28441d4c1f admin admin] PATH_INFO: `/v3/roles` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:55.527913 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b5609a-6cef-4c07-97ab-8b28441d4c1f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:55.529518 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b5609a-6cef-4c07-97ab-8b28441d4c1f admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:55.540323 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1035/2096] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 10:48:55 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 10:48:55.544982 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c750975-9ade-49b2-8e52-d15a37e3786c admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:55.557051 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c750975-9ade-49b2-8e52-d15a37e3786c admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:55.562008 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f17e72-bef5-4622-8c19-7c7f85b62e7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:55.562662 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f17e72-bef5-4622-8c19-7c7f85b62e7c admin admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:55.562975 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f17e72-bef5-4622-8c19-7c7f85b62e7c admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:55.563254 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f17e72-bef5-4622-8c19-7c7f85b62e7c admin admin] PATH_INFO: `/v3/roles` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:55.563950 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f17e72-bef5-4622-8c19-7c7f85b62e7c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:55.565556 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f17e72-bef5-4622-8c19-7c7f85b62e7c admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:55.575747 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1062/2097] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 10:48:55 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 10:48:55.579932 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b5609a-6cef-4c07-97ab-8b28441d4c1f admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:55.590393 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b5609a-6cef-4c07-97ab-8b28441d4c1f admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:55.595125 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648d2a2b-2dab-4957-af0d-5d0dcbe9f56d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:55.595785 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648d2a2b-2dab-4957-af0d-5d0dcbe9f56d admin admin] REQUEST_METHOD: `PUT` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:55.596128 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648d2a2b-2dab-4957-af0d-5d0dcbe9f56d admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:55.596469 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648d2a2b-2dab-4957-af0d-5d0dcbe9f56d admin admin] PATH_INFO: `/v3/projects/5303a1c35c634f8aafc353bcea56e9d6/users/881090625557466bb2b9c8fe01f1e4c2/roles/4c9fc0b3baed441da2126be41cc6b04a` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:55.612446 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648d2a2b-2dab-4957-af0d-5d0dcbe9f56d admin admin] RBAC: Authorizing `identity:create_grant(project_id=5303a1c35c634f8aafc353bcea56e9d6, user_id=881090625557466bb2b9c8fe01f1e4c2, role_id=4c9fc0b3baed441da2126be41cc6b04a)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:55.615118 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648d2a2b-2dab-4957-af0d-5d0dcbe9f56d admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:55.629138 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1036/2098] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 10:48:55 2022] PUT /identity/v3/projects/5303a1c35c634f8aafc353bcea56e9d6/users/881090625557466bb2b9c8fe01f1e4c2/roles/4c9fc0b3baed441da2126be41cc6b04a => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:48:55.640297 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bceb55b4-1427-43ac-80b9-195b0e8e0580 None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:55.640620 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bceb55b4-1427-43ac-80b9-195b0e8e0580 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:55.640979 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bceb55b4-1427-43ac-80b9-195b0e8e0580 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:55.673706 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-bceb55b4-1427-43ac-80b9-195b0e8e0580 None None] MFA Rules not processed for user `881090625557466bb2b9c8fe01f1e4c2`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:48:55.675465 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-bceb55b4-1427-43ac-80b9-195b0e8e0580 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:55.740634 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1063/2099] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:48:55 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:48:55.745253 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648d2a2b-2dab-4957-af0d-5d0dcbe9f56d admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:55.819513 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648d2a2b-2dab-4957-af0d-5d0dcbe9f56d admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:55.825851 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d154af2b-79b8-4fd4-ba19-f586f4b6410e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:55.826525 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d154af2b-79b8-4fd4-ba19-f586f4b6410e admin admin] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:55.826839 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d154af2b-79b8-4fd4-ba19-f586f4b6410e admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:55.827115 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d154af2b-79b8-4fd4-ba19-f586f4b6410e admin admin] PATH_INFO: `/v3/projects` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:55.827781 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d154af2b-79b8-4fd4-ba19-f586f4b6410e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:55.830529 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d154af2b-79b8-4fd4-ba19-f586f4b6410e admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:55.851182 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:48:55.864936 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1037/2100] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Jul 27 10:48:55 2022] POST /identity/v3/projects => generated 385 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 10:48:55.869154 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bceb55b4-1427-43ac-80b9-195b0e8e0580 None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:55.880250 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bceb55b4-1427-43ac-80b9-195b0e8e0580 None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:55.942576 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665fffc3-5380-4280-9196-5d3f782236fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:55.942576 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665fffc3-5380-4280-9196-5d3f782236fe admin admin] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:55.942576 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665fffc3-5380-4280-9196-5d3f782236fe admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:55.942576 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665fffc3-5380-4280-9196-5d3f782236fe admin admin] PATH_INFO: `/v3/users` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:55.943308 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665fffc3-5380-4280-9196-5d3f782236fe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:55.943308 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665fffc3-5380-4280-9196-5d3f782236fe admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:55.967504 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1064/2101] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Jul 27 10:48:55 2022] POST /identity/v3/users => generated 299 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 10:48:56.032010 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d154af2b-79b8-4fd4-ba19-f586f4b6410e admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:56.042120 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d154af2b-79b8-4fd4-ba19-f586f4b6410e admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:56.046699 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7daa2d99-0248-45b4-87ac-eef18f77958c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:56.047319 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7daa2d99-0248-45b4-87ac-eef18f77958c admin admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:56.047596 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7daa2d99-0248-45b4-87ac-eef18f77958c admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:56.047868 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7daa2d99-0248-45b4-87ac-eef18f77958c admin admin] PATH_INFO: `/v3/roles` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:56.048558 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7daa2d99-0248-45b4-87ac-eef18f77958c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:56.050147 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7daa2d99-0248-45b4-87ac-eef18f77958c admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:56.062058 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1038/2102] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 10:48:56 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 10:48:56.066942 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665fffc3-5380-4280-9196-5d3f782236fe admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:56.082880 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665fffc3-5380-4280-9196-5d3f782236fe admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:56.091014 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1976ca-c2fc-47af-a775-bf8678d64490 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:56.091706 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1976ca-c2fc-47af-a775-bf8678d64490 admin admin] REQUEST_METHOD: `PUT` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:56.091985 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1976ca-c2fc-47af-a775-bf8678d64490 admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:56.092257 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1976ca-c2fc-47af-a775-bf8678d64490 admin admin] PATH_INFO: `/v3/projects/cd226bf11d0c4401bdb59b79b302cafe/users/a98a07c7b7da4c8bbbdb045245d62b72/roles/21156560eb6b436bb567a40b4976f3f4` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:56.114000 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1976ca-c2fc-47af-a775-bf8678d64490 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd226bf11d0c4401bdb59b79b302cafe, user_id=a98a07c7b7da4c8bbbdb045245d62b72, role_id=21156560eb6b436bb567a40b4976f3f4)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:56.119069 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1976ca-c2fc-47af-a775-bf8678d64490 admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:56.133891 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1065/2103] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 10:48:56 2022] PUT /identity/v3/projects/cd226bf11d0c4401bdb59b79b302cafe/users/a98a07c7b7da4c8bbbdb045245d62b72/roles/21156560eb6b436bb567a40b4976f3f4 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:48:56.139046 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7daa2d99-0248-45b4-87ac-eef18f77958c admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:56.149510 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7daa2d99-0248-45b4-87ac-eef18f77958c admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:56.255435 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26895f86-0e12-4991-9698-f57d2be9ff6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:56.256150 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26895f86-0e12-4991-9698-f57d2be9ff6b admin admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:56.256520 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26895f86-0e12-4991-9698-f57d2be9ff6b admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:56.256707 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26895f86-0e12-4991-9698-f57d2be9ff6b admin admin] PATH_INFO: `/v3/roles` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:56.257484 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26895f86-0e12-4991-9698-f57d2be9ff6b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:56.264209 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26895f86-0e12-4991-9698-f57d2be9ff6b admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:56.282653 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1039/2104] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 10:48:56 2022] GET /identity/v3/roles => generated 1420 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 10:48:56.284969 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1976ca-c2fc-47af-a775-bf8678d64490 admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:56.301547 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1976ca-c2fc-47af-a775-bf8678d64490 admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:56.307225 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc6f770-57f6-4fa3-b26e-aeb1f3e9f61d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:56.308651 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc6f770-57f6-4fa3-b26e-aeb1f3e9f61d admin admin] REQUEST_METHOD: `PUT` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:56.309263 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc6f770-57f6-4fa3-b26e-aeb1f3e9f61d admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:56.309622 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc6f770-57f6-4fa3-b26e-aeb1f3e9f61d admin admin] PATH_INFO: `/v3/projects/cd226bf11d0c4401bdb59b79b302cafe/users/a98a07c7b7da4c8bbbdb045245d62b72/roles/21156560eb6b436bb567a40b4976f3f4` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:56.320764 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc6f770-57f6-4fa3-b26e-aeb1f3e9f61d admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd226bf11d0c4401bdb59b79b302cafe, user_id=a98a07c7b7da4c8bbbdb045245d62b72, role_id=21156560eb6b436bb567a40b4976f3f4)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:56.326090 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc6f770-57f6-4fa3-b26e-aeb1f3e9f61d admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:56.334904 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:48:56.340479 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1066/2105] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 10:48:56 2022] PUT /identity/v3/projects/cd226bf11d0c4401bdb59b79b302cafe/users/a98a07c7b7da4c8bbbdb045245d62b72/roles/21156560eb6b436bb567a40b4976f3f4 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:48:56.351641 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a99b0b-fcf0-45d4-ad67-4aa9bc0304ce None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:56.351922 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a99b0b-fcf0-45d4-ad67-4aa9bc0304ce None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:56.352239 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a99b0b-fcf0-45d4-ad67-4aa9bc0304ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:56.404625 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-39a99b0b-fcf0-45d4-ad67-4aa9bc0304ce None None] MFA Rules not processed for user `a98a07c7b7da4c8bbbdb045245d62b72`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:48:56.406312 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-39a99b0b-fcf0-45d4-ad67-4aa9bc0304ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:56.451420 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc6f770-57f6-4fa3-b26e-aeb1f3e9f61d admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:56.453240 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-1fc6f770-57f6-4fa3-b26e-aeb1f3e9f61d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:56.513382 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1040/2106] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:48:56 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:48:56.516629 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc6f770-57f6-4fa3-b26e-aeb1f3e9f61d admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:56.520796 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfdb7c1-3346-411e-95bb-08ebc3d58406 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '864d35f268cf45148756109759bb2743', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:56.521428 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfdb7c1-3346-411e-95bb-08ebc3d58406 service cinder] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:56.521739 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfdb7c1-3346-411e-95bb-08ebc3d58406 service cinder] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:56.522022 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfdb7c1-3346-411e-95bb-08ebc3d58406 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:56.524020 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-8cfdb7c1-3346-411e-95bb-08ebc3d58406 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:56.524750 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a99b0b-fcf0-45d4-ad67-4aa9bc0304ce None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:56.534124 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a99b0b-fcf0-45d4-ad67-4aa9bc0304ce None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:56.537875 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1d3c9e-13ae-4ecb-9e14-d73bac211b9a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '864d35f268cf45148756109759bb2743', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:56.538484 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1d3c9e-13ae-4ecb-9e14-d73bac211b9a service cinder] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:56.538797 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1d3c9e-13ae-4ecb-9e14-d73bac211b9a service cinder] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:56.539076 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1d3c9e-13ae-4ecb-9e14-d73bac211b9a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:56.596316 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfdb7c1-3346-411e-95bb-08ebc3d58406 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:56.598618 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfdb7c1-3346-411e-95bb-08ebc3d58406 service cinder] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:56.601645 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1d3c9e-13ae-4ecb-9e14-d73bac211b9a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:56.603184 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1d3c9e-13ae-4ecb-9e14-d73bac211b9a service cinder] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:56.613085 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1067/2107] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:48:56 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 10:48:56.615080 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1041/2108] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:48:56 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 10:48:59.797061 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfdb7c1-3346-411e-95bb-08ebc3d58406 service cinder] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:48:59.805442 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfdb7c1-3346-411e-95bb-08ebc3d58406 service cinder] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:48:59.808998 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab36add2-9b7d-4f83-a7df-568144bad90b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '864d35f268cf45148756109759bb2743', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:48:59.809598 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab36add2-9b7d-4f83-a7df-568144bad90b service cinder] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:48:59.809878 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab36add2-9b7d-4f83-a7df-568144bad90b service cinder] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:48:59.810151 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab36add2-9b7d-4f83-a7df-568144bad90b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:48:59.811879 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-ab36add2-9b7d-4f83-a7df-568144bad90b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:48:59.868899 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab36add2-9b7d-4f83-a7df-568144bad90b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:48:59.870333 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab36add2-9b7d-4f83-a7df-568144bad90b service cinder] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:48:59.880382 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1068/2109] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:48:59 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 10:49:00.504728 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1d3c9e-13ae-4ecb-9e14-d73bac211b9a service cinder] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:49:00.508727 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:49:00.515826 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1d3c9e-13ae-4ecb-9e14-d73bac211b9a service cinder] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:49:00.521259 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbf87c8-6dd2-4c1c-b5f8-7571cb5582e1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '864d35f268cf45148756109759bb2743', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:49:00.521885 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbf87c8-6dd2-4c1c-b5f8-7571cb5582e1 service cinder] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:49:00.522165 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbf87c8-6dd2-4c1c-b5f8-7571cb5582e1 service cinder] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:49:00.522468 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbf87c8-6dd2-4c1c-b5f8-7571cb5582e1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:49:00.644237 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbf87c8-6dd2-4c1c-b5f8-7571cb5582e1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:49:00.644237 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbf87c8-6dd2-4c1c-b5f8-7571cb5582e1 service cinder] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:49:00.645395 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1042/2110] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:49:00 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 10:49:00.877433 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab36add2-9b7d-4f83-a7df-568144bad90b service cinder] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:49:00.878950 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-ab36add2-9b7d-4f83-a7df-568144bad90b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:49:00.932814 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:49:00.951005 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab36add2-9b7d-4f83-a7df-568144bad90b service cinder] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:49:00.956090 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098283f8-d71b-4b07-b47d-4eae47519886 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:49:00.956770 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098283f8-d71b-4b07-b47d-4eae47519886 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:49:00.957056 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098283f8-d71b-4b07-b47d-4eae47519886 admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:49:00.957336 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098283f8-d71b-4b07-b47d-4eae47519886 admin admin] PATH_INFO: `/v3/users/2f81981f4c184f6cb9bef624d7096b35` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:49:00.958543 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098283f8-d71b-4b07-b47d-4eae47519886 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2f81981f4c184f6cb9bef624d7096b35)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:49:00.960742 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098283f8-d71b-4b07-b47d-4eae47519886 admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:49:01.013571 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-098283f8-d71b-4b07-b47d-4eae47519886 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2f81981f4c184f6cb9bef624d7096b35'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:49:01.021540 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-098283f8-d71b-4b07-b47d-4eae47519886 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2f81981f4c184f6cb9bef624d7096b35'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:49:01.029243 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-098283f8-d71b-4b07-b47d-4eae47519886 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2f81981f4c184f6cb9bef624d7096b35'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:49:01.030003 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-098283f8-d71b-4b07-b47d-4eae47519886 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2f81981f4c184f6cb9bef624d7096b35'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:49:01.030676 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-098283f8-d71b-4b07-b47d-4eae47519886 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2f81981f4c184f6cb9bef624d7096b35'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:49:01.054353 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1069/2111] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Jul 27 10:49:00 2022] DELETE /identity/v3/users/2f81981f4c184f6cb9bef624d7096b35 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:49:01.059101 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0f9490-bfb0-48a7-ac52-98b49aec0ef2 None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:49:01.059419 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0f9490-bfb0-48a7-ac52-98b49aec0ef2 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:49:01.059771 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0f9490-bfb0-48a7-ac52-98b49aec0ef2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:49:01.081767 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-8f0f9490-bfb0-48a7-ac52-98b49aec0ef2 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:49:01.082998 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-8f0f9490-bfb0-48a7-ac52-98b49aec0ef2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:49:01.149177 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1043/2112] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:49:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:49:01.161772 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098283f8-d71b-4b07-b47d-4eae47519886 admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:49:01.163408 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-098283f8-d71b-4b07-b47d-4eae47519886 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:49:01.235606 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098283f8-d71b-4b07-b47d-4eae47519886 admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:49:01.239890 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c253e6-b1e3-4555-b60b-538ce3f6bc91 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7922d0a1a7c4e808d0b2a80dca0da40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:49:01.240519 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c253e6-b1e3-4555-b60b-538ce3f6bc91 service neutron] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:49:01.240798 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c253e6-b1e3-4555-b60b-538ce3f6bc91 service neutron] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:49:01.241169 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c253e6-b1e3-4555-b60b-538ce3f6bc91 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:49:01.252260 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c253e6-b1e3-4555-b60b-538ce3f6bc91 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:49:01.253908 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c253e6-b1e3-4555-b60b-538ce3f6bc91 service neutron] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:49:01.277389 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1070/2113] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:49:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 10:49:01.517317 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0f9490-bfb0-48a7-ac52-98b49aec0ef2 None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:49:01.518990 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-8f0f9490-bfb0-48a7-ac52-98b49aec0ef2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:49:01.599386 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0f9490-bfb0-48a7-ac52-98b49aec0ef2 None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:49:01.617237 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ce6823-796f-4ea8-aaa2-e06f5362b145 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '864d35f268cf45148756109759bb2743', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:49:01.618172 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ce6823-796f-4ea8-aaa2-e06f5362b145 service cinder] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:49:01.618463 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ce6823-796f-4ea8-aaa2-e06f5362b145 service cinder] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:49:01.619438 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ce6823-796f-4ea8-aaa2-e06f5362b145 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:49:01.622346 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-34ce6823-796f-4ea8-aaa2-e06f5362b145 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:49:01.637871 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c253e6-b1e3-4555-b60b-538ce3f6bc91 service neutron] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:49:01.640472 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-80c253e6-b1e3-4555-b60b-538ce3f6bc91 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:49:01.657356 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c253e6-b1e3-4555-b60b-538ce3f6bc91 service neutron] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:49:01.667041 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dec2579-915c-492e-bb03-f1874ad610ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:49:01.669371 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dec2579-915c-492e-bb03-f1874ad610ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:49:01.669673 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dec2579-915c-492e-bb03-f1874ad610ce admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:49:01.670721 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dec2579-915c-492e-bb03-f1874ad610ce admin admin] PATH_INFO: `/v3/projects/c3719baac0324707b90c1823647f047e` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:49:01.674017 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dec2579-915c-492e-bb03-f1874ad610ce admin admin] RBAC: Authorizing `identity:delete_project(project_id=c3719baac0324707b90c1823647f047e)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:49:01.680224 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dec2579-915c-492e-bb03-f1874ad610ce admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:49:01.700614 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:49:01.719505 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ce6823-796f-4ea8-aaa2-e06f5362b145 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:49:01.721526 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ce6823-796f-4ea8-aaa2-e06f5362b145 service cinder] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:49:01.740540 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1044/2114] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:49:01 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 10:49:01.775584 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-6dec2579-915c-492e-bb03-f1874ad610ce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c3719baac0324707b90c1823647f047e', 'request_id': 'req-6dec2579-915c-492e-bb03-f1874ad610ce'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:49:01.785575 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-6dec2579-915c-492e-bb03-f1874ad610ce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c3719baac0324707b90c1823647f047e', 'request_id': 'req-6dec2579-915c-492e-bb03-f1874ad610ce'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:49:01.797312 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-6dec2579-915c-492e-bb03-f1874ad610ce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c3719baac0324707b90c1823647f047e', 'request_id': 'req-6dec2579-915c-492e-bb03-f1874ad610ce'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:49:01.806084 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:49:01.822260 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-6dec2579-915c-492e-bb03-f1874ad610ce admin admin] The token cache is being invalidate because project c3719baac0324707b90c1823647f047e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86827) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 10:49:01.822566 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-6dec2579-915c-492e-bb03-f1874ad610ce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:49:01.824041 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-6dec2579-915c-492e-bb03-f1874ad610ce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:49:01.826002 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1071/2115] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Jul 27 10:49:01 2022] DELETE /identity/v3/projects/c3719baac0324707b90c1823647f047e => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:49:01.830591 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ce6823-796f-4ea8-aaa2-e06f5362b145 service cinder] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:49:01.842950 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ce6823-796f-4ea8-aaa2-e06f5362b145 service cinder] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:49:01.914636 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6754a12-2b1d-4436-8e0a-d7f08996ee61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:49:01.916209 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6754a12-2b1d-4436-8e0a-d7f08996ee61 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:49:01.916507 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6754a12-2b1d-4436-8e0a-d7f08996ee61 admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:49:01.916792 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6754a12-2b1d-4436-8e0a-d7f08996ee61 admin admin] PATH_INFO: `/v3/users/c5c353bffcb143f59e9c0936cd7c4cdf` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:49:01.919466 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6754a12-2b1d-4436-8e0a-d7f08996ee61 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c5c353bffcb143f59e9c0936cd7c4cdf)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:49:01.921356 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6754a12-2b1d-4436-8e0a-d7f08996ee61 admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:49:02.010970 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-f6754a12-2b1d-4436-8e0a-d7f08996ee61 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c5c353bffcb143f59e9c0936cd7c4cdf'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:49:02.018729 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-f6754a12-2b1d-4436-8e0a-d7f08996ee61 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c5c353bffcb143f59e9c0936cd7c4cdf'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:49:02.028085 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-f6754a12-2b1d-4436-8e0a-d7f08996ee61 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c5c353bffcb143f59e9c0936cd7c4cdf'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:49:02.031305 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-f6754a12-2b1d-4436-8e0a-d7f08996ee61 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c5c353bffcb143f59e9c0936cd7c4cdf'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:49:02.031968 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-f6754a12-2b1d-4436-8e0a-d7f08996ee61 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c5c353bffcb143f59e9c0936cd7c4cdf'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:49:02.062577 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1045/2116] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Jul 27 10:49:01 2022] DELETE /identity/v3/users/c5c353bffcb143f59e9c0936cd7c4cdf => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:49:02.379001 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dec2579-915c-492e-bb03-f1874ad610ce admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:49:02.380628 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-6dec2579-915c-492e-bb03-f1874ad610ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:49:02.393823 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dec2579-915c-492e-bb03-f1874ad610ce admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:49:02.452211 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6754a12-2b1d-4436-8e0a-d7f08996ee61 admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:49:02.453812 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-f6754a12-2b1d-4436-8e0a-d7f08996ee61 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:49:02.467327 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f55c7c1-be91-4327-932a-56410445c82d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:49:02.468017 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f55c7c1-be91-4327-932a-56410445c82d admin admin] REQUEST_METHOD: `DELETE` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:49:02.468306 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f55c7c1-be91-4327-932a-56410445c82d admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:49:02.468582 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f55c7c1-be91-4327-932a-56410445c82d admin admin] PATH_INFO: `/v3/users/a23c4f92439442c5a70ea683ff2ca0bc` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:49:02.469759 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f55c7c1-be91-4327-932a-56410445c82d admin admin] RBAC: Authorizing `identity:delete_user(user_id=a23c4f92439442c5a70ea683ff2ca0bc)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:49:02.471760 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f55c7c1-be91-4327-932a-56410445c82d admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:49:02.519612 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-2f55c7c1-be91-4327-932a-56410445c82d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a23c4f92439442c5a70ea683ff2ca0bc'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:49:02.527408 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-2f55c7c1-be91-4327-932a-56410445c82d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a23c4f92439442c5a70ea683ff2ca0bc'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:49:02.530334 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6754a12-2b1d-4436-8e0a-d7f08996ee61 admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:49:02.535078 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-2f55c7c1-be91-4327-932a-56410445c82d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a23c4f92439442c5a70ea683ff2ca0bc'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:49:02.535879 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d470be-aff5-4657-8234-1f394b0359c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:49:02.536285 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-2f55c7c1-be91-4327-932a-56410445c82d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a23c4f92439442c5a70ea683ff2ca0bc'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:49:02.536514 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d470be-aff5-4657-8234-1f394b0359c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:49:02.536805 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d470be-aff5-4657-8234-1f394b0359c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:49:02.536909 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-2f55c7c1-be91-4327-932a-56410445c82d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a23c4f92439442c5a70ea683ff2ca0bc'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:49:02.537066 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d470be-aff5-4657-8234-1f394b0359c0 admin admin] PATH_INFO: `/v3/projects/d10675b447774d24940bd500f20ffba2` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:49:02.538267 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d470be-aff5-4657-8234-1f394b0359c0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d10675b447774d24940bd500f20ffba2)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:49:02.540172 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d470be-aff5-4657-8234-1f394b0359c0 admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:49:02.551625 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:49:02.556156 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1072/2117] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Jul 27 10:49:02 2022] DELETE /identity/v3/users/a23c4f92439442c5a70ea683ff2ca0bc => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:49:02.560572 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6475b414-393d-4992-a43f-712cbc060270 None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:49:02.560995 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6475b414-393d-4992-a43f-712cbc060270 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:49:02.561283 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6475b414-393d-4992-a43f-712cbc060270 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:49:02.581988 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-6475b414-393d-4992-a43f-712cbc060270 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:49:02.583252 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-6475b414-393d-4992-a43f-712cbc060270 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:49:02.592912 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-29d470be-aff5-4657-8234-1f394b0359c0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd10675b447774d24940bd500f20ffba2', 'request_id': 'req-29d470be-aff5-4657-8234-1f394b0359c0'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:49:02.599498 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-29d470be-aff5-4657-8234-1f394b0359c0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd10675b447774d24940bd500f20ffba2', 'request_id': 'req-29d470be-aff5-4657-8234-1f394b0359c0'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:49:02.608818 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-29d470be-aff5-4657-8234-1f394b0359c0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd10675b447774d24940bd500f20ffba2', 'request_id': 'req-29d470be-aff5-4657-8234-1f394b0359c0'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:49:02.625947 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-29d470be-aff5-4657-8234-1f394b0359c0 admin admin] The token cache is being invalidate because project d10675b447774d24940bd500f20ffba2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86828) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 10:49:02.625947 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-29d470be-aff5-4657-8234-1f394b0359c0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:49:02.626963 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-29d470be-aff5-4657-8234-1f394b0359c0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:49:02.628758 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1046/2118] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Jul 27 10:49:02 2022] DELETE /identity/v3/projects/d10675b447774d24940bd500f20ffba2 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 10:49:02.633266 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d470be-aff5-4657-8234-1f394b0359c0 admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:49:02.634683 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-29d470be-aff5-4657-8234-1f394b0359c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:49:02.657088 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1073/2119] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:49:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 10:49:02.668536 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6475b414-393d-4992-a43f-712cbc060270 None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:49:02.683414 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-6475b414-393d-4992-a43f-712cbc060270 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:49:02.700297 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:49:02.707319 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d470be-aff5-4657-8234-1f394b0359c0 admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:49:02.715547 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af61a115-efb5-4949-bf13-92aaba44e45b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:49:02.716324 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af61a115-efb5-4949-bf13-92aaba44e45b admin admin] REQUEST_METHOD: `DELETE` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:49:02.717038 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af61a115-efb5-4949-bf13-92aaba44e45b admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:49:02.717542 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af61a115-efb5-4949-bf13-92aaba44e45b admin admin] PATH_INFO: `/v3/users/ccd048bb2e0847ada5f6aed923672dea` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:49:02.719176 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af61a115-efb5-4949-bf13-92aaba44e45b admin admin] RBAC: Authorizing `identity:delete_user(user_id=ccd048bb2e0847ada5f6aed923672dea)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:49:02.721224 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af61a115-efb5-4949-bf13-92aaba44e45b admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:49:02.752521 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6475b414-393d-4992-a43f-712cbc060270 None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:49:02.756953 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5462884e-2a76-4bdb-bec3-6b968cf65a77 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7922d0a1a7c4e808d0b2a80dca0da40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:49:02.757846 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5462884e-2a76-4bdb-bec3-6b968cf65a77 service neutron] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:49:02.758105 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5462884e-2a76-4bdb-bec3-6b968cf65a77 service neutron] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:49:02.758404 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5462884e-2a76-4bdb-bec3-6b968cf65a77 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:49:02.768676 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5462884e-2a76-4bdb-bec3-6b968cf65a77 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:49:02.770111 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5462884e-2a76-4bdb-bec3-6b968cf65a77 service neutron] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:49:02.780523 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-af61a115-efb5-4949-bf13-92aaba44e45b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ccd048bb2e0847ada5f6aed923672dea'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:49:02.781409 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1074/2120] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:49:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 10:49:02.788418 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-af61a115-efb5-4949-bf13-92aaba44e45b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ccd048bb2e0847ada5f6aed923672dea'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:49:02.799210 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-af61a115-efb5-4949-bf13-92aaba44e45b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ccd048bb2e0847ada5f6aed923672dea'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:49:02.800041 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-af61a115-efb5-4949-bf13-92aaba44e45b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ccd048bb2e0847ada5f6aed923672dea'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:49:02.800757 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-af61a115-efb5-4949-bf13-92aaba44e45b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ccd048bb2e0847ada5f6aed923672dea'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:49:02.820996 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1047/2121] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Jul 27 10:49:02 2022] DELETE /identity/v3/users/ccd048bb2e0847ada5f6aed923672dea => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 10:49:03.205932 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5462884e-2a76-4bdb-bec3-6b968cf65a77 service neutron] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:49:03.220681 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5462884e-2a76-4bdb-bec3-6b968cf65a77 service neutron] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:49:03.408587 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52523e8-1f5c-42fb-b59c-3ec3e2804c82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:49:03.409977 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52523e8-1f5c-42fb-b59c-3ec3e2804c82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:49:03.410553 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52523e8-1f5c-42fb-b59c-3ec3e2804c82 admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:49:03.411568 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52523e8-1f5c-42fb-b59c-3ec3e2804c82 admin admin] PATH_INFO: `/v3/projects/21c10e481b60469da940254d0ea10fb4` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:49:03.412824 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52523e8-1f5c-42fb-b59c-3ec3e2804c82 admin admin] RBAC: Authorizing `identity:delete_project(project_id=21c10e481b60469da940254d0ea10fb4)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:49:03.415039 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52523e8-1f5c-42fb-b59c-3ec3e2804c82 admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:49:03.476305 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-b52523e8-1f5c-42fb-b59c-3ec3e2804c82 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '21c10e481b60469da940254d0ea10fb4', 'request_id': 'req-b52523e8-1f5c-42fb-b59c-3ec3e2804c82'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:49:03.483580 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af61a115-efb5-4949-bf13-92aaba44e45b admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:49:03.485160 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-af61a115-efb5-4949-bf13-92aaba44e45b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:49:03.488848 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-b52523e8-1f5c-42fb-b59c-3ec3e2804c82 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '21c10e481b60469da940254d0ea10fb4', 'request_id': 'req-b52523e8-1f5c-42fb-b59c-3ec3e2804c82'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:49:03.498019 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-b52523e8-1f5c-42fb-b59c-3ec3e2804c82 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '21c10e481b60469da940254d0ea10fb4', 'request_id': 'req-b52523e8-1f5c-42fb-b59c-3ec3e2804c82'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:49:03.506677 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af61a115-efb5-4949-bf13-92aaba44e45b admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:49:03.516902 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-b52523e8-1f5c-42fb-b59c-3ec3e2804c82 admin admin] The token cache is being invalidate because project 21c10e481b60469da940254d0ea10fb4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86827) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 10:49:03.517199 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-b52523e8-1f5c-42fb-b59c-3ec3e2804c82 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:49:03.517864 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-b52523e8-1f5c-42fb-b59c-3ec3e2804c82 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:49:03.519731 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1075/2122] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Jul 27 10:49:03 2022] DELETE /identity/v3/projects/21c10e481b60469da940254d0ea10fb4 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:49:03.526636 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0435f0a0-a3ec-4532-857a-ac4471b56aa6 None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:49:03.527010 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0435f0a0-a3ec-4532-857a-ac4471b56aa6 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:49:03.527290 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0435f0a0-a3ec-4532-857a-ac4471b56aa6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:49:03.531149 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:49:03.550315 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-0435f0a0-a3ec-4532-857a-ac4471b56aa6 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:49:03.551662 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-0435f0a0-a3ec-4532-857a-ac4471b56aa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:49:03.576957 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f90ee6-77c7-49d0-b7d1-3741d3f6e009 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:49:03.577613 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f90ee6-77c7-49d0-b7d1-3741d3f6e009 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:49:03.577898 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f90ee6-77c7-49d0-b7d1-3741d3f6e009 admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:49:03.578176 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f90ee6-77c7-49d0-b7d1-3741d3f6e009 admin admin] PATH_INFO: `/v3/projects/33ddf88108e94513b7d66246ff7315cd` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:49:03.579371 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f90ee6-77c7-49d0-b7d1-3741d3f6e009 admin admin] RBAC: Authorizing `identity:delete_project(project_id=33ddf88108e94513b7d66246ff7315cd)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:49:03.581291 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f90ee6-77c7-49d0-b7d1-3741d3f6e009 admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:49:03.594022 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:49:03.614210 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1076/2123] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:49:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:49:03.623196 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125123f5-b313-4ca5-87e4-0e0db4d82ada None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:49:03.623554 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125123f5-b313-4ca5-87e4-0e0db4d82ada None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:49:03.623848 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125123f5-b313-4ca5-87e4-0e0db4d82ada None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:49:03.633430 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-80f90ee6-77c7-49d0-b7d1-3741d3f6e009 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '33ddf88108e94513b7d66246ff7315cd', 'request_id': 'req-80f90ee6-77c7-49d0-b7d1-3741d3f6e009'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:49:03.639773 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-80f90ee6-77c7-49d0-b7d1-3741d3f6e009 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '33ddf88108e94513b7d66246ff7315cd', 'request_id': 'req-80f90ee6-77c7-49d0-b7d1-3741d3f6e009'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:49:03.643485 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-125123f5-b313-4ca5-87e4-0e0db4d82ada None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:49:03.644765 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-125123f5-b313-4ca5-87e4-0e0db4d82ada None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:49:03.648445 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-80f90ee6-77c7-49d0-b7d1-3741d3f6e009 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '33ddf88108e94513b7d66246ff7315cd', 'request_id': 'req-80f90ee6-77c7-49d0-b7d1-3741d3f6e009'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:49:03.660605 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-80f90ee6-77c7-49d0-b7d1-3741d3f6e009 admin admin] The token cache is being invalidate because project 33ddf88108e94513b7d66246ff7315cd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86828) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 10:49:03.661197 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-80f90ee6-77c7-49d0-b7d1-3741d3f6e009 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:49:03.662043 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-80f90ee6-77c7-49d0-b7d1-3741d3f6e009 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:49:03.664772 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1048/2124] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Jul 27 10:49:03 2022] DELETE /identity/v3/projects/33ddf88108e94513b7d66246ff7315cd => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 10:49:03.669459 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f90ee6-77c7-49d0-b7d1-3741d3f6e009 admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:49:03.672099 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-80f90ee6-77c7-49d0-b7d1-3741d3f6e009 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:49:03.706319 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1077/2125] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:49:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:49:03.710497 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125123f5-b313-4ca5-87e4-0e0db4d82ada None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:49:03.719877 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125123f5-b313-4ca5-87e4-0e0db4d82ada None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:49:03.724881 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5941673a-816c-4f09-aa05-ea2b42beb136 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:49:03.725490 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5941673a-816c-4f09-aa05-ea2b42beb136 admin admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:49:03.725825 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5941673a-816c-4f09-aa05-ea2b42beb136 admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:49:03.726113 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5941673a-816c-4f09-aa05-ea2b42beb136 admin admin] PATH_INFO: `/v3/domains` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:49:03.726848 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5941673a-816c-4f09-aa05-ea2b42beb136 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86827) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:49:03.727164 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5941673a-816c-4f09-aa05-ea2b42beb136 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:49:03.728797 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5941673a-816c-4f09-aa05-ea2b42beb136 admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:49:03.740020 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f90ee6-77c7-49d0-b7d1-3741d3f6e009 admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:49:03.741500 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1078/2126] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Jul 27 10:49:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 10:49:03.745243 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62071417-4a99-481e-b396-1e91ae1026c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:49:03.745430 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5941673a-816c-4f09-aa05-ea2b42beb136 admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:49:03.746101 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62071417-4a99-481e-b396-1e91ae1026c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:49:03.746454 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62071417-4a99-481e-b396-1e91ae1026c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:49:03.746832 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62071417-4a99-481e-b396-1e91ae1026c5 admin admin] PATH_INFO: `/v3/users/a874864af3b2496b8dbf359b39f51459` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:49:03.748097 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62071417-4a99-481e-b396-1e91ae1026c5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a874864af3b2496b8dbf359b39f51459)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:49:03.750136 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62071417-4a99-481e-b396-1e91ae1026c5 admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:49:03.755263 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5941673a-816c-4f09-aa05-ea2b42beb136 admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:49:03.760576 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f2cef0-2410-4861-9016-177aaad3df0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:49:03.761175 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f2cef0-2410-4861-9016-177aaad3df0c admin admin] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:49:03.761459 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f2cef0-2410-4861-9016-177aaad3df0c admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:49:03.762802 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f2cef0-2410-4861-9016-177aaad3df0c admin admin] PATH_INFO: `/v3/projects` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:49:03.763473 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f2cef0-2410-4861-9016-177aaad3df0c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:49:03.765351 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f2cef0-2410-4861-9016-177aaad3df0c admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:49:03.795291 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1079/2127] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Jul 27 10:49:03 2022] POST /identity/v3/projects => generated 397 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 10:49:03.799360 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f2cef0-2410-4861-9016-177aaad3df0c admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:49:03.802142 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-62071417-4a99-481e-b396-1e91ae1026c5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a874864af3b2496b8dbf359b39f51459'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:49:03.807158 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:49:03.811454 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-62071417-4a99-481e-b396-1e91ae1026c5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a874864af3b2496b8dbf359b39f51459'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:49:03.819664 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-62071417-4a99-481e-b396-1e91ae1026c5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a874864af3b2496b8dbf359b39f51459'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:49:03.820306 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-62071417-4a99-481e-b396-1e91ae1026c5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a874864af3b2496b8dbf359b39f51459'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:49:03.820941 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-62071417-4a99-481e-b396-1e91ae1026c5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a874864af3b2496b8dbf359b39f51459'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:49:03.838700 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1049/2128] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Jul 27 10:49:03 2022] DELETE /identity/v3/users/a874864af3b2496b8dbf359b39f51459 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:49:03.865691 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f2cef0-2410-4861-9016-177aaad3df0c admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:49:03.932431 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299618a2-9df0-41b1-bed7-700f15cb6cff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:49:03.933094 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299618a2-9df0-41b1-bed7-700f15cb6cff admin admin] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:49:03.933380 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299618a2-9df0-41b1-bed7-700f15cb6cff admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:49:03.933662 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299618a2-9df0-41b1-bed7-700f15cb6cff admin admin] PATH_INFO: `/v3/users` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:49:03.934336 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299618a2-9df0-41b1-bed7-700f15cb6cff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:49:03.936390 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299618a2-9df0-41b1-bed7-700f15cb6cff admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:49:03.946737 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:49:03.966235 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1080/2129] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Jul 27 10:49:03 2022] POST /identity/v3/users => generated 299 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 10:49:03.967073 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62071417-4a99-481e-b396-1e91ae1026c5 admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:49:03.968466 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-62071417-4a99-481e-b396-1e91ae1026c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:49:03.982621 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62071417-4a99-481e-b396-1e91ae1026c5 admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:49:03.989067 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba159235-ab2e-4f45-b596-4ab04fc103fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:49:03.989707 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba159235-ab2e-4f45-b596-4ab04fc103fc admin admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:49:03.989990 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba159235-ab2e-4f45-b596-4ab04fc103fc admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:49:03.995369 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba159235-ab2e-4f45-b596-4ab04fc103fc admin admin] PATH_INFO: `/v3/roles` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:49:03.996807 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba159235-ab2e-4f45-b596-4ab04fc103fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:49:03.998777 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba159235-ab2e-4f45-b596-4ab04fc103fc admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:49:04.009735 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1050/2130] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 10:49:03 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 10:49:04.015301 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299618a2-9df0-41b1-bed7-700f15cb6cff admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:49:04.026442 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299618a2-9df0-41b1-bed7-700f15cb6cff admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:49:04.031726 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c81200-d871-4e40-9e80-631fed3e19bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:49:04.032355 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c81200-d871-4e40-9e80-631fed3e19bb admin admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:49:04.032643 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c81200-d871-4e40-9e80-631fed3e19bb admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:49:04.032923 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c81200-d871-4e40-9e80-631fed3e19bb admin admin] PATH_INFO: `/v3/roles` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:49:04.033626 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c81200-d871-4e40-9e80-631fed3e19bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:49:04.035700 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c81200-d871-4e40-9e80-631fed3e19bb admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:49:04.046111 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1081/2131] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 10:49:04 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 10:49:04.050320 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba159235-ab2e-4f45-b596-4ab04fc103fc admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:49:04.060484 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba159235-ab2e-4f45-b596-4ab04fc103fc admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:49:04.065427 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdc5261-1360-44fb-b5f4-85eb0b780ab8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:49:04.066109 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdc5261-1360-44fb-b5f4-85eb0b780ab8 admin admin] REQUEST_METHOD: `PUT` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:49:04.066397 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdc5261-1360-44fb-b5f4-85eb0b780ab8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:49:04.066707 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdc5261-1360-44fb-b5f4-85eb0b780ab8 admin admin] PATH_INFO: `/v3/projects/82101cb97faa4613a27f33a4871b7d9f/users/19317a4ce6fe430abc4cf2d10a0f5772/roles/4c9fc0b3baed441da2126be41cc6b04a` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:49:04.082590 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdc5261-1360-44fb-b5f4-85eb0b780ab8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=82101cb97faa4613a27f33a4871b7d9f, user_id=19317a4ce6fe430abc4cf2d10a0f5772, role_id=4c9fc0b3baed441da2126be41cc6b04a)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:49:04.085536 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdc5261-1360-44fb-b5f4-85eb0b780ab8 admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:49:04.100317 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1051/2132] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 10:49:04 2022] PUT /identity/v3/projects/82101cb97faa4613a27f33a4871b7d9f/users/19317a4ce6fe430abc4cf2d10a0f5772/roles/4c9fc0b3baed441da2126be41cc6b04a => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 10:49:04.111924 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6701e6-f630-49f7-a77f-c1b697356a57 None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:49:04.112247 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6701e6-f630-49f7-a77f-c1b697356a57 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:49:04.112526 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6701e6-f630-49f7-a77f-c1b697356a57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:49:04.126927 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdc5261-1360-44fb-b5f4-85eb0b780ab8 admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:49:04.128620 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-efdc5261-1360-44fb-b5f4-85eb0b780ab8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:49:04.169479 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-af6701e6-f630-49f7-a77f-c1b697356a57 None None] MFA Rules not processed for user `19317a4ce6fe430abc4cf2d10a0f5772`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:49:04.171241 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-af6701e6-f630-49f7-a77f-c1b697356a57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:49:04.200893 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdc5261-1360-44fb-b5f4-85eb0b780ab8 admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:49:04.205644 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440627c1-64d5-4bb7-b880-aba49ac0afd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:49:04.206291 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440627c1-64d5-4bb7-b880-aba49ac0afd5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:49:04.206577 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440627c1-64d5-4bb7-b880-aba49ac0afd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:49:04.206885 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440627c1-64d5-4bb7-b880-aba49ac0afd5 admin admin] PATH_INFO: `/v3/projects/3bac58bc4270449eae3fa7b995fcf34e` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:49:04.208049 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440627c1-64d5-4bb7-b880-aba49ac0afd5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3bac58bc4270449eae3fa7b995fcf34e)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:49:04.210001 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440627c1-64d5-4bb7-b880-aba49ac0afd5 admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:49:04.247500 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1082/2133] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:49:04 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:49:04.258678 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-440627c1-64d5-4bb7-b880-aba49ac0afd5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3bac58bc4270449eae3fa7b995fcf34e', 'request_id': 'req-440627c1-64d5-4bb7-b880-aba49ac0afd5'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:49:04.259282 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6701e6-f630-49f7-a77f-c1b697356a57 None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:49:04.260673 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-af6701e6-f630-49f7-a77f-c1b697356a57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:49:04.265822 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-440627c1-64d5-4bb7-b880-aba49ac0afd5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3bac58bc4270449eae3fa7b995fcf34e', 'request_id': 'req-440627c1-64d5-4bb7-b880-aba49ac0afd5'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:49:04.275038 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-440627c1-64d5-4bb7-b880-aba49ac0afd5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3bac58bc4270449eae3fa7b995fcf34e', 'request_id': 'req-440627c1-64d5-4bb7-b880-aba49ac0afd5'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:49:04.286984 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-440627c1-64d5-4bb7-b880-aba49ac0afd5 admin admin] The token cache is being invalidate because project 3bac58bc4270449eae3fa7b995fcf34e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86828) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 10:49:04.287273 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-440627c1-64d5-4bb7-b880-aba49ac0afd5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:49:04.288033 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-440627c1-64d5-4bb7-b880-aba49ac0afd5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:49:04.289591 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1052/2134] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Jul 27 10:49:04 2022] DELETE /identity/v3/projects/3bac58bc4270449eae3fa7b995fcf34e => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:49:04.293531 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440627c1-64d5-4bb7-b880-aba49ac0afd5 admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:49:04.295029 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-440627c1-64d5-4bb7-b880-aba49ac0afd5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:49:04.319907 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6701e6-f630-49f7-a77f-c1b697356a57 None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:49:04.324141 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:49:04.363809 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440627c1-64d5-4bb7-b880-aba49ac0afd5 admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:49:04.369321 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3b8093-1e50-417d-aa74-eb656b381ae2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:49:04.369970 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3b8093-1e50-417d-aa74-eb656b381ae2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:49:04.370805 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3b8093-1e50-417d-aa74-eb656b381ae2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:49:04.371065 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53444f8-b536-4a17-88df-e811ead74ecd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '864d35f268cf45148756109759bb2743', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:49:04.371205 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3b8093-1e50-417d-aa74-eb656b381ae2 admin admin] PATH_INFO: `/v3/users/d58d04a3e9d84ef5b621cd7e30d1d72b` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:49:04.371696 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53444f8-b536-4a17-88df-e811ead74ecd service cinder] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:49:04.372006 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53444f8-b536-4a17-88df-e811ead74ecd service cinder] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:49:04.372250 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53444f8-b536-4a17-88df-e811ead74ecd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:49:04.373790 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3b8093-1e50-417d-aa74-eb656b381ae2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d58d04a3e9d84ef5b621cd7e30d1d72b)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:49:04.374032 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-d53444f8-b536-4a17-88df-e811ead74ecd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:49:04.375816 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3b8093-1e50-417d-aa74-eb656b381ae2 admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:49:04.437628 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:49:04.443223 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-db3b8093-1e50-417d-aa74-eb656b381ae2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd58d04a3e9d84ef5b621cd7e30d1d72b'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:49:04.451588 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-db3b8093-1e50-417d-aa74-eb656b381ae2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd58d04a3e9d84ef5b621cd7e30d1d72b'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:49:04.463542 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-db3b8093-1e50-417d-aa74-eb656b381ae2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd58d04a3e9d84ef5b621cd7e30d1d72b'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:49:04.464210 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53444f8-b536-4a17-88df-e811ead74ecd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:49:04.464348 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-db3b8093-1e50-417d-aa74-eb656b381ae2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd58d04a3e9d84ef5b621cd7e30d1d72b'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:49:04.464957 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-db3b8093-1e50-417d-aa74-eb656b381ae2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd58d04a3e9d84ef5b621cd7e30d1d72b'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:49:04.465672 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53444f8-b536-4a17-88df-e811ead74ecd service cinder] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:49:04.475299 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1083/2135] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:49:04 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 10:49:04.493199 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1053/2136] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Jul 27 10:49:04 2022] DELETE /identity/v3/users/d58d04a3e9d84ef5b621cd7e30d1d72b => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 10:49:04.817235 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53444f8-b536-4a17-88df-e811ead74ecd service cinder] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:49:04.836636 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53444f8-b536-4a17-88df-e811ead74ecd service cinder] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:49:04.916951 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e4c8be-25db-4aa1-ae6f-aca40e426a8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:49:04.918512 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e4c8be-25db-4aa1-ae6f-aca40e426a8c admin admin] REQUEST_METHOD: `DELETE` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:49:04.918899 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e4c8be-25db-4aa1-ae6f-aca40e426a8c admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:49:04.919220 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e4c8be-25db-4aa1-ae6f-aca40e426a8c admin admin] PATH_INFO: `/v3/projects/c5cdef5e9520477b820bb8df4bcbd720` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:49:04.920471 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e4c8be-25db-4aa1-ae6f-aca40e426a8c admin admin] RBAC: Authorizing `identity:delete_project(project_id=c5cdef5e9520477b820bb8df4bcbd720)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:49:04.922421 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e4c8be-25db-4aa1-ae6f-aca40e426a8c admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:49:04.986657 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-85e4c8be-25db-4aa1-ae6f-aca40e426a8c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c5cdef5e9520477b820bb8df4bcbd720', 'request_id': 'req-85e4c8be-25db-4aa1-ae6f-aca40e426a8c'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:49:04.994198 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-85e4c8be-25db-4aa1-ae6f-aca40e426a8c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c5cdef5e9520477b820bb8df4bcbd720', 'request_id': 'req-85e4c8be-25db-4aa1-ae6f-aca40e426a8c'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:49:05.003044 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-85e4c8be-25db-4aa1-ae6f-aca40e426a8c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c5cdef5e9520477b820bb8df4bcbd720', 'request_id': 'req-85e4c8be-25db-4aa1-ae6f-aca40e426a8c'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:49:05.013593 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-85e4c8be-25db-4aa1-ae6f-aca40e426a8c admin admin] The token cache is being invalidate because project c5cdef5e9520477b820bb8df4bcbd720 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86827) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 10:49:05.013841 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-85e4c8be-25db-4aa1-ae6f-aca40e426a8c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:49:05.014469 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-85e4c8be-25db-4aa1-ae6f-aca40e426a8c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:49:05.016012 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1084/2137] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Jul 27 10:49:04 2022] DELETE /identity/v3/projects/c5cdef5e9520477b820bb8df4bcbd720 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:49:05.021801 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251e70ca-acb9-453e-b7f8-bfb39e2d8220 None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:49:05.022261 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251e70ca-acb9-453e-b7f8-bfb39e2d8220 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:49:05.022356 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251e70ca-acb9-453e-b7f8-bfb39e2d8220 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:49:05.041694 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-251e70ca-acb9-453e-b7f8-bfb39e2d8220 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:49:05.043429 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-251e70ca-acb9-453e-b7f8-bfb39e2d8220 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:49:05.100837 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1054/2138] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:49:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:49:05.111020 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1048b4-5432-47a1-93f5-637f609357e4 None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:49:05.111310 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1048b4-5432-47a1-93f5-637f609357e4 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:49:05.111586 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1048b4-5432-47a1-93f5-637f609357e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:49:05.131503 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-1f1048b4-5432-47a1-93f5-637f609357e4 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:49:05.132695 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-1f1048b4-5432-47a1-93f5-637f609357e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:49:05.140473 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1085/2139] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:49:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:49:05.144589 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251e70ca-acb9-453e-b7f8-bfb39e2d8220 None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:49:05.154845 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251e70ca-acb9-453e-b7f8-bfb39e2d8220 None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:49:05.159466 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0964f873-d985-48ba-9c7e-b5ab9809be0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:49:05.160073 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0964f873-d985-48ba-9c7e-b5ab9809be0a admin admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:49:05.160360 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0964f873-d985-48ba-9c7e-b5ab9809be0a admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:49:05.160636 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0964f873-d985-48ba-9c7e-b5ab9809be0a admin admin] PATH_INFO: `/v3/domains` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:49:05.161337 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0964f873-d985-48ba-9c7e-b5ab9809be0a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86828) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:49:05.161642 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0964f873-d985-48ba-9c7e-b5ab9809be0a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:49:05.163280 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0964f873-d985-48ba-9c7e-b5ab9809be0a admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:49:05.175474 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1055/2140] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Jul 27 10:49:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 10:49:05.179576 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1048b4-5432-47a1-93f5-637f609357e4 None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:49:05.189141 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1048b4-5432-47a1-93f5-637f609357e4 None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:49:05.193727 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669f3273-1340-4540-a8bb-808a9fe63bf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:49:05.194328 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669f3273-1340-4540-a8bb-808a9fe63bf1 admin admin] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:49:05.194620 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669f3273-1340-4540-a8bb-808a9fe63bf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:49:05.194911 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669f3273-1340-4540-a8bb-808a9fe63bf1 admin admin] PATH_INFO: `/v3/projects` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:49:05.195566 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669f3273-1340-4540-a8bb-808a9fe63bf1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:49:05.197450 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669f3273-1340-4540-a8bb-808a9fe63bf1 admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:49:05.225238 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1086/2141] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Jul 27 10:49:05 2022] POST /identity/v3/projects => generated 373 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 10:49:05.230345 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0964f873-d985-48ba-9c7e-b5ab9809be0a admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:49:05.240449 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0964f873-d985-48ba-9c7e-b5ab9809be0a admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:49:05.268733 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:49:05.297592 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be399f70-efd3-4975-a7fa-e8f91b05afc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:49:05.298204 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be399f70-efd3-4975-a7fa-e8f91b05afc1 admin admin] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:49:05.298485 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be399f70-efd3-4975-a7fa-e8f91b05afc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:49:05.298787 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be399f70-efd3-4975-a7fa-e8f91b05afc1 admin admin] PATH_INFO: `/v3/users` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:49:05.299453 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be399f70-efd3-4975-a7fa-e8f91b05afc1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:49:05.301358 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be399f70-efd3-4975-a7fa-e8f91b05afc1 admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:49:05.321763 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1056/2142] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Jul 27 10:49:05 2022] POST /identity/v3/users => generated 287 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 10:49:05.325913 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669f3273-1340-4540-a8bb-808a9fe63bf1 admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:49:05.336273 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669f3273-1340-4540-a8bb-808a9fe63bf1 admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:49:05.341406 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb7eb1b-5dba-4caf-a38e-b84e0fa48d28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:49:05.342018 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb7eb1b-5dba-4caf-a38e-b84e0fa48d28 admin admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:49:05.342304 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb7eb1b-5dba-4caf-a38e-b84e0fa48d28 admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:49:05.342584 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb7eb1b-5dba-4caf-a38e-b84e0fa48d28 admin admin] PATH_INFO: `/v3/roles` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:49:05.343379 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb7eb1b-5dba-4caf-a38e-b84e0fa48d28 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:49:05.344978 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb7eb1b-5dba-4caf-a38e-b84e0fa48d28 admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:49:05.355490 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1087/2143] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 10:49:05 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 10:49:05.359105 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be399f70-efd3-4975-a7fa-e8f91b05afc1 admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:49:05.368894 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be399f70-efd3-4975-a7fa-e8f91b05afc1 admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:49:05.373495 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8daac35e-430c-4dd4-94e9-5b1d40b51f24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:49:05.374110 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8daac35e-430c-4dd4-94e9-5b1d40b51f24 admin admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:49:05.374397 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8daac35e-430c-4dd4-94e9-5b1d40b51f24 admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:49:05.374716 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8daac35e-430c-4dd4-94e9-5b1d40b51f24 admin admin] PATH_INFO: `/v3/roles` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:49:05.375410 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8daac35e-430c-4dd4-94e9-5b1d40b51f24 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:49:05.377271 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8daac35e-430c-4dd4-94e9-5b1d40b51f24 admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:49:05.387691 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1057/2144] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 10:49:05 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 10:49:05.391590 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb7eb1b-5dba-4caf-a38e-b84e0fa48d28 admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:49:05.402038 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb7eb1b-5dba-4caf-a38e-b84e0fa48d28 admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:49:05.406717 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa54c1fa-e6d4-4ef8-b7e6-37681bcb8a24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:49:05.407376 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa54c1fa-e6d4-4ef8-b7e6-37681bcb8a24 admin admin] REQUEST_METHOD: `PUT` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:49:05.407660 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa54c1fa-e6d4-4ef8-b7e6-37681bcb8a24 admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:49:05.407940 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa54c1fa-e6d4-4ef8-b7e6-37681bcb8a24 admin admin] PATH_INFO: `/v3/projects/39d5c31eff78482abf021b2d8da046f8/users/478146bf73944604bdd9f4129fdc98f0/roles/4c9fc0b3baed441da2126be41cc6b04a` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:49:05.423517 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa54c1fa-e6d4-4ef8-b7e6-37681bcb8a24 admin admin] RBAC: Authorizing `identity:create_grant(project_id=39d5c31eff78482abf021b2d8da046f8, user_id=478146bf73944604bdd9f4129fdc98f0, role_id=4c9fc0b3baed441da2126be41cc6b04a)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:49:05.426185 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa54c1fa-e6d4-4ef8-b7e6-37681bcb8a24 admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:49:05.441361 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1088/2145] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 10:49:05 2022] PUT /identity/v3/projects/39d5c31eff78482abf021b2d8da046f8/users/478146bf73944604bdd9f4129fdc98f0/roles/4c9fc0b3baed441da2126be41cc6b04a => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 10:49:05.453179 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4ec22f-3e28-4d21-b93c-22ba6a36efee None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:49:05.453531 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4ec22f-3e28-4d21-b93c-22ba6a36efee None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:49:05.453817 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4ec22f-3e28-4d21-b93c-22ba6a36efee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:49:05.482066 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-7c4ec22f-3e28-4d21-b93c-22ba6a36efee None None] MFA Rules not processed for user `478146bf73944604bdd9f4129fdc98f0`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:49:05.483317 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-7c4ec22f-3e28-4d21-b93c-22ba6a36efee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:49:05.546761 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1058/2146] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:49:05 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:49:05.558400 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa54c1fa-e6d4-4ef8-b7e6-37681bcb8a24 admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:49:05.560131 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-aa54c1fa-e6d4-4ef8-b7e6-37681bcb8a24 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:49:05.566623 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:49:05.620319 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa54c1fa-e6d4-4ef8-b7e6-37681bcb8a24 admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:49:05.624013 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770f03f4-2778-491d-8fad-3bc7a5111e37 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '864d35f268cf45148756109759bb2743', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:49:05.624636 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770f03f4-2778-491d-8fad-3bc7a5111e37 service cinder] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:49:05.624921 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770f03f4-2778-491d-8fad-3bc7a5111e37 service cinder] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:49:05.625198 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770f03f4-2778-491d-8fad-3bc7a5111e37 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:49:05.634104 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770f03f4-2778-491d-8fad-3bc7a5111e37 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:49:05.635582 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770f03f4-2778-491d-8fad-3bc7a5111e37 service cinder] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:49:05.645096 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1089/2147] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:49:05 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 10:49:05.667553 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4ec22f-3e28-4d21-b93c-22ba6a36efee None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:49:05.743108 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4ec22f-3e28-4d21-b93c-22ba6a36efee None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:49:05.747770 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae791c0-ed22-4344-ab66-d217bac42d51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:49:05.748412 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae791c0-ed22-4344-ab66-d217bac42d51 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:49:05.748696 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae791c0-ed22-4344-ab66-d217bac42d51 admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:49:05.748978 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae791c0-ed22-4344-ab66-d217bac42d51 admin admin] PATH_INFO: `/v3/users/478146bf73944604bdd9f4129fdc98f0` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:49:05.750157 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae791c0-ed22-4344-ab66-d217bac42d51 admin admin] RBAC: Authorizing `identity:delete_user(user_id=478146bf73944604bdd9f4129fdc98f0)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:49:05.752366 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae791c0-ed22-4344-ab66-d217bac42d51 admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:49:05.796079 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=91,keystone:INSERT=3,keystone:DELETE=6 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:49:05.803389 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-9ae791c0-ed22-4344-ab66-d217bac42d51 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '478146bf73944604bdd9f4129fdc98f0'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:49:05.813022 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-9ae791c0-ed22-4344-ab66-d217bac42d51 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '478146bf73944604bdd9f4129fdc98f0'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:49:05.820147 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-9ae791c0-ed22-4344-ab66-d217bac42d51 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '478146bf73944604bdd9f4129fdc98f0'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:49:05.820924 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-9ae791c0-ed22-4344-ab66-d217bac42d51 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '478146bf73944604bdd9f4129fdc98f0'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:49:05.821587 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-9ae791c0-ed22-4344-ab66-d217bac42d51 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '478146bf73944604bdd9f4129fdc98f0'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:49:05.842592 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1059/2148] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Jul 27 10:49:05 2022] DELETE /identity/v3/users/478146bf73944604bdd9f4129fdc98f0 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:49:05.847023 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8884ced-0da7-458c-8cd5-f0afdbd44e77 None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:49:05.847395 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8884ced-0da7-458c-8cd5-f0afdbd44e77 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:49:05.847816 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8884ced-0da7-458c-8cd5-f0afdbd44e77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:49:05.867063 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-b8884ced-0da7-458c-8cd5-f0afdbd44e77 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:49:05.868313 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-b8884ced-0da7-458c-8cd5-f0afdbd44e77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:49:05.933962 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1090/2149] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:49:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 10:49:05.945312 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae791c0-ed22-4344-ab66-d217bac42d51 admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:49:05.946844 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-9ae791c0-ed22-4344-ab66-d217bac42d51 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:49:06.006970 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae791c0-ed22-4344-ab66-d217bac42d51 admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:49:06.010563 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532acc17-8f16-4426-bff6-cff6364d465e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7922d0a1a7c4e808d0b2a80dca0da40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:49:06.011790 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532acc17-8f16-4426-bff6-cff6364d465e service neutron] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:49:06.012069 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532acc17-8f16-4426-bff6-cff6364d465e service neutron] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:49:06.012341 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532acc17-8f16-4426-bff6-cff6364d465e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:49:06.021431 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532acc17-8f16-4426-bff6-cff6364d465e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:49:06.023032 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532acc17-8f16-4426-bff6-cff6364d465e service neutron] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:49:06.035534 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1060/2150] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:49:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 10:49:06.312268 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8884ced-0da7-458c-8cd5-f0afdbd44e77 None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:49:06.313695 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-b8884ced-0da7-458c-8cd5-f0afdbd44e77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:49:06.325910 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8884ced-0da7-458c-8cd5-f0afdbd44e77 None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:49:06.330564 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a93bcf5-5a2b-4738-b67f-8b5208cdb172 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:49:06.331352 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a93bcf5-5a2b-4738-b67f-8b5208cdb172 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:49:06.331709 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a93bcf5-5a2b-4738-b67f-8b5208cdb172 admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:49:06.331992 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a93bcf5-5a2b-4738-b67f-8b5208cdb172 admin admin] PATH_INFO: `/v3/projects/39d5c31eff78482abf021b2d8da046f8` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:49:06.333184 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a93bcf5-5a2b-4738-b67f-8b5208cdb172 admin admin] RBAC: Authorizing `identity:delete_project(project_id=39d5c31eff78482abf021b2d8da046f8)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:49:06.335129 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a93bcf5-5a2b-4738-b67f-8b5208cdb172 admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:49:06.384912 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-8a93bcf5-5a2b-4738-b67f-8b5208cdb172 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '39d5c31eff78482abf021b2d8da046f8', 'request_id': 'req-8a93bcf5-5a2b-4738-b67f-8b5208cdb172'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:49:06.391014 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-8a93bcf5-5a2b-4738-b67f-8b5208cdb172 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '39d5c31eff78482abf021b2d8da046f8', 'request_id': 'req-8a93bcf5-5a2b-4738-b67f-8b5208cdb172'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:49:06.398632 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-8a93bcf5-5a2b-4738-b67f-8b5208cdb172 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '39d5c31eff78482abf021b2d8da046f8', 'request_id': 'req-8a93bcf5-5a2b-4738-b67f-8b5208cdb172'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:49:06.409019 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-8a93bcf5-5a2b-4738-b67f-8b5208cdb172 admin admin] The token cache is being invalidate because project 39d5c31eff78482abf021b2d8da046f8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86827) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 10:49:06.409302 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-8a93bcf5-5a2b-4738-b67f-8b5208cdb172 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:49:06.409959 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-8a93bcf5-5a2b-4738-b67f-8b5208cdb172 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:49:06.411808 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1091/2151] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Jul 27 10:49:06 2022] DELETE /identity/v3/projects/39d5c31eff78482abf021b2d8da046f8 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:49:06.416826 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98d0245-8ae3-4a2d-8755-55725d788851 None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:49:06.417640 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98d0245-8ae3-4a2d-8755-55725d788851 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:49:06.417924 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98d0245-8ae3-4a2d-8755-55725d788851 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:49:06.438703 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-e98d0245-8ae3-4a2d-8755-55725d788851 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:49:06.439891 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-e98d0245-8ae3-4a2d-8755-55725d788851 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:49:06.516446 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1061/2152] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:49:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 10:49:06.523949 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f86d63-49c0-48e4-9c9f-437c06b435fd None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:49:06.524489 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f86d63-49c0-48e4-9c9f-437c06b435fd None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:49:06.524649 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f86d63-49c0-48e4-9c9f-437c06b435fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:49:06.546378 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-64f86d63-49c0-48e4-9c9f-437c06b435fd None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:49:06.547686 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-64f86d63-49c0-48e4-9c9f-437c06b435fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:49:06.558645 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1092/2153] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:49:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:49:06.561202 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98d0245-8ae3-4a2d-8755-55725d788851 None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:49:06.572093 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98d0245-8ae3-4a2d-8755-55725d788851 None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:49:06.577059 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778e178a-b037-4a8c-b280-459865daa59a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:49:06.577836 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778e178a-b037-4a8c-b280-459865daa59a admin admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:49:06.578218 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778e178a-b037-4a8c-b280-459865daa59a admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:49:06.578627 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778e178a-b037-4a8c-b280-459865daa59a admin admin] PATH_INFO: `/v3/domains` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:49:06.579442 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778e178a-b037-4a8c-b280-459865daa59a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86828) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:49:06.579851 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778e178a-b037-4a8c-b280-459865daa59a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:49:06.582066 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778e178a-b037-4a8c-b280-459865daa59a admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:49:06.598666 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1062/2154] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Jul 27 10:49:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:49:06.600493 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f86d63-49c0-48e4-9c9f-437c06b435fd None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:49:06.610760 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f86d63-49c0-48e4-9c9f-437c06b435fd None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:49:06.615450 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08022540-7218-45ee-826b-1c1895e35e6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:49:06.616218 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08022540-7218-45ee-826b-1c1895e35e6d admin admin] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:49:06.616499 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08022540-7218-45ee-826b-1c1895e35e6d admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:49:06.616775 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08022540-7218-45ee-826b-1c1895e35e6d admin admin] PATH_INFO: `/v3/projects` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:49:06.617445 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08022540-7218-45ee-826b-1c1895e35e6d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:49:06.620941 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08022540-7218-45ee-826b-1c1895e35e6d admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:49:06.659873 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1093/2155] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Jul 27 10:49:06 2022] POST /identity/v3/projects => generated 383 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 10:49:06.661199 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778e178a-b037-4a8c-b280-459865daa59a admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:49:06.673785 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778e178a-b037-4a8c-b280-459865daa59a admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:49:06.728334 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:49:06.748162 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59fff72-ab50-4330-af52-145e9a7b0565 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:49:06.748890 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59fff72-ab50-4330-af52-145e9a7b0565 admin admin] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:49:06.749242 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59fff72-ab50-4330-af52-145e9a7b0565 admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:49:06.749670 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59fff72-ab50-4330-af52-145e9a7b0565 admin admin] PATH_INFO: `/v3/users` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:49:06.750418 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59fff72-ab50-4330-af52-145e9a7b0565 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:49:06.753210 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59fff72-ab50-4330-af52-145e9a7b0565 admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:49:06.782775 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1063/2156] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Jul 27 10:49:06 2022] POST /identity/v3/users => generated 292 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 10:49:06.784251 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08022540-7218-45ee-826b-1c1895e35e6d admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:49:06.790195 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:49:06.798431 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08022540-7218-45ee-826b-1c1895e35e6d admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:49:06.810489 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f8eebc-7f15-4417-8699-4b747226068b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:49:06.811170 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f8eebc-7f15-4417-8699-4b747226068b admin admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:49:06.811456 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f8eebc-7f15-4417-8699-4b747226068b admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:49:06.811732 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f8eebc-7f15-4417-8699-4b747226068b admin admin] PATH_INFO: `/v3/roles` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:49:06.812429 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f8eebc-7f15-4417-8699-4b747226068b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:49:06.814055 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f8eebc-7f15-4417-8699-4b747226068b admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:49:06.828446 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1094/2157] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 10:49:06 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 10:49:06.829162 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59fff72-ab50-4330-af52-145e9a7b0565 admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:49:06.845422 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59fff72-ab50-4330-af52-145e9a7b0565 admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:49:06.851464 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec33e55-41dd-45ae-b8a9-4ffdc5c55c34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:49:06.852514 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec33e55-41dd-45ae-b8a9-4ffdc5c55c34 admin admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:49:06.852514 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec33e55-41dd-45ae-b8a9-4ffdc5c55c34 admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:49:06.852793 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec33e55-41dd-45ae-b8a9-4ffdc5c55c34 admin admin] PATH_INFO: `/v3/roles` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:49:06.853471 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec33e55-41dd-45ae-b8a9-4ffdc5c55c34 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:49:06.855748 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec33e55-41dd-45ae-b8a9-4ffdc5c55c34 admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:49:06.867655 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1064/2158] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 10:49:06 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 10:49:06.871118 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f8eebc-7f15-4417-8699-4b747226068b admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:49:06.883788 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f8eebc-7f15-4417-8699-4b747226068b admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:49:06.890384 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d72564-54f3-4a12-9333-b5cc72b0a44d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:49:06.891108 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d72564-54f3-4a12-9333-b5cc72b0a44d admin admin] REQUEST_METHOD: `PUT` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:49:06.891674 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d72564-54f3-4a12-9333-b5cc72b0a44d admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:49:06.891834 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d72564-54f3-4a12-9333-b5cc72b0a44d admin admin] PATH_INFO: `/v3/projects/3c288c531b5f431d819c0eafff040145/users/516a5f83148d4ccb885b3a901f14f8fa/roles/4c9fc0b3baed441da2126be41cc6b04a` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:49:06.918218 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d72564-54f3-4a12-9333-b5cc72b0a44d admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c288c531b5f431d819c0eafff040145, user_id=516a5f83148d4ccb885b3a901f14f8fa, role_id=4c9fc0b3baed441da2126be41cc6b04a)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:49:06.920963 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d72564-54f3-4a12-9333-b5cc72b0a44d admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:49:06.940593 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1095/2159] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 10:49:06 2022] PUT /identity/v3/projects/3c288c531b5f431d819c0eafff040145/users/516a5f83148d4ccb885b3a901f14f8fa/roles/4c9fc0b3baed441da2126be41cc6b04a => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:49:06.958024 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cdec13-2222-4052-acd1-a3eaa46ea644 None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:49:06.958437 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cdec13-2222-4052-acd1-a3eaa46ea644 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:49:06.958933 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cdec13-2222-4052-acd1-a3eaa46ea644 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:49:07.034104 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-93cdec13-2222-4052-acd1-a3eaa46ea644 None None] MFA Rules not processed for user `516a5f83148d4ccb885b3a901f14f8fa`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:49:07.036280 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-93cdec13-2222-4052-acd1-a3eaa46ea644 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:49:07.131089 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1065/2160] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:49:06 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:49:07.140226 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d72564-54f3-4a12-9333-b5cc72b0a44d admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:49:07.143171 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-89d72564-54f3-4a12-9333-b5cc72b0a44d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:49:07.236491 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d72564-54f3-4a12-9333-b5cc72b0a44d admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:49:07.241767 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da3f01c-ef8f-4ae8-92bd-adcebf163078 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '864d35f268cf45148756109759bb2743', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:49:07.242498 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da3f01c-ef8f-4ae8-92bd-adcebf163078 service cinder] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:49:07.242904 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da3f01c-ef8f-4ae8-92bd-adcebf163078 service cinder] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:49:07.243236 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da3f01c-ef8f-4ae8-92bd-adcebf163078 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:49:07.256199 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da3f01c-ef8f-4ae8-92bd-adcebf163078 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:49:07.259047 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da3f01c-ef8f-4ae8-92bd-adcebf163078 service cinder] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:49:07.272243 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1096/2161] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:49:07 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 10:49:07.620135 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cdec13-2222-4052-acd1-a3eaa46ea644 None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:49:07.621700 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-93cdec13-2222-4052-acd1-a3eaa46ea644 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:49:07.717182 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cdec13-2222-4052-acd1-a3eaa46ea644 None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:49:07.721888 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7d9204-0de0-4621-8562-62c49b561775 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:49:07.722474 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7d9204-0de0-4621-8562-62c49b561775 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:49:07.722800 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7d9204-0de0-4621-8562-62c49b561775 admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:49:07.723082 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7d9204-0de0-4621-8562-62c49b561775 admin admin] PATH_INFO: `/v3/users/5e9fec0a854b4a9cab8799dd51ed17ed` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:49:07.724275 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7d9204-0de0-4621-8562-62c49b561775 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5e9fec0a854b4a9cab8799dd51ed17ed)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:49:07.726488 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7d9204-0de0-4621-8562-62c49b561775 admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:49:07.775694 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-fb7d9204-0de0-4621-8562-62c49b561775 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5e9fec0a854b4a9cab8799dd51ed17ed'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:49:07.783299 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-fb7d9204-0de0-4621-8562-62c49b561775 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5e9fec0a854b4a9cab8799dd51ed17ed'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:49:07.785776 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:49:07.791166 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-fb7d9204-0de0-4621-8562-62c49b561775 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5e9fec0a854b4a9cab8799dd51ed17ed'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:49:07.791917 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-fb7d9204-0de0-4621-8562-62c49b561775 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5e9fec0a854b4a9cab8799dd51ed17ed'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:49:07.793143 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-fb7d9204-0de0-4621-8562-62c49b561775 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5e9fec0a854b4a9cab8799dd51ed17ed'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:49:07.811650 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1066/2162] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Jul 27 10:49:07 2022] DELETE /identity/v3/users/5e9fec0a854b4a9cab8799dd51ed17ed => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:49:07.817357 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445f2508-6b50-4281-99ce-d083f9adabf6 None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:49:07.817848 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445f2508-6b50-4281-99ce-d083f9adabf6 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:49:07.817944 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445f2508-6b50-4281-99ce-d083f9adabf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:49:07.837264 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-445f2508-6b50-4281-99ce-d083f9adabf6 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:49:07.838459 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-445f2508-6b50-4281-99ce-d083f9adabf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:49:07.895626 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1097/2163] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:49:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 10:49:07.906116 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7d9204-0de0-4621-8562-62c49b561775 admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:49:07.907578 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-fb7d9204-0de0-4621-8562-62c49b561775 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:49:07.965383 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7d9204-0de0-4621-8562-62c49b561775 admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:49:07.969147 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79165f05-bfc8-4756-bab3-5a89f6dc5530 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7922d0a1a7c4e808d0b2a80dca0da40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:49:07.969792 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79165f05-bfc8-4756-bab3-5a89f6dc5530 service neutron] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:49:07.970080 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79165f05-bfc8-4756-bab3-5a89f6dc5530 service neutron] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:49:07.970389 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79165f05-bfc8-4756-bab3-5a89f6dc5530 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:49:07.979433 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79165f05-bfc8-4756-bab3-5a89f6dc5530 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:49:07.980847 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79165f05-bfc8-4756-bab3-5a89f6dc5530 service neutron] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:49:07.990470 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1067/2164] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:49:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 10:49:08.060653 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445f2508-6b50-4281-99ce-d083f9adabf6 None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:49:08.064631 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-445f2508-6b50-4281-99ce-d083f9adabf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:49:08.080765 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445f2508-6b50-4281-99ce-d083f9adabf6 None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:49:08.086573 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622c31ad-a276-4706-a05b-3ccebc95ea4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:49:08.087352 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622c31ad-a276-4706-a05b-3ccebc95ea4a admin admin] REQUEST_METHOD: `DELETE` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:49:08.087747 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622c31ad-a276-4706-a05b-3ccebc95ea4a admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:49:08.088179 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622c31ad-a276-4706-a05b-3ccebc95ea4a admin admin] PATH_INFO: `/v3/users/67469ca501314670b39dd957b5f8646e` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:49:08.089737 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622c31ad-a276-4706-a05b-3ccebc95ea4a admin admin] RBAC: Authorizing `identity:delete_user(user_id=67469ca501314670b39dd957b5f8646e)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:49:08.091815 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622c31ad-a276-4706-a05b-3ccebc95ea4a admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:49:08.145585 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-622c31ad-a276-4706-a05b-3ccebc95ea4a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '67469ca501314670b39dd957b5f8646e'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:49:08.169500 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-622c31ad-a276-4706-a05b-3ccebc95ea4a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '67469ca501314670b39dd957b5f8646e'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:49:08.179296 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-622c31ad-a276-4706-a05b-3ccebc95ea4a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '67469ca501314670b39dd957b5f8646e'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:49:08.180353 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-622c31ad-a276-4706-a05b-3ccebc95ea4a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '67469ca501314670b39dd957b5f8646e'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:49:08.181338 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-622c31ad-a276-4706-a05b-3ccebc95ea4a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '67469ca501314670b39dd957b5f8646e'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:49:08.189255 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:49:08.207546 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1098/2165] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Jul 27 10:49:08 2022] DELETE /identity/v3/users/67469ca501314670b39dd957b5f8646e => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:49:08.212051 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4378186-9d39-4f00-85a1-fec8860af381 None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:49:08.212454 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4378186-9d39-4f00-85a1-fec8860af381 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:49:08.212608 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4378186-9d39-4f00-85a1-fec8860af381 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:49:08.238456 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-e4378186-9d39-4f00-85a1-fec8860af381 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:49:08.239711 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-e4378186-9d39-4f00-85a1-fec8860af381 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:49:08.311062 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1068/2166] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:49:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:49:08.322150 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622c31ad-a276-4706-a05b-3ccebc95ea4a admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:49:08.324264 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-622c31ad-a276-4706-a05b-3ccebc95ea4a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:49:08.340966 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4378186-9d39-4f00-85a1-fec8860af381 None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:49:08.342418 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-e4378186-9d39-4f00-85a1-fec8860af381 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:49:08.361567 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4378186-9d39-4f00-85a1-fec8860af381 None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:49:08.367089 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5399e4c3-4c4b-416b-bc68-96e51df0050c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:49:08.367811 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5399e4c3-4c4b-416b-bc68-96e51df0050c admin admin] REQUEST_METHOD: `DELETE` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:49:08.368161 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5399e4c3-4c4b-416b-bc68-96e51df0050c admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:49:08.368508 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5399e4c3-4c4b-416b-bc68-96e51df0050c admin admin] PATH_INFO: `/v3/projects/687ae3b37ca843aabd40a3218beb0092` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:49:08.369911 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5399e4c3-4c4b-416b-bc68-96e51df0050c admin admin] RBAC: Authorizing `identity:delete_project(project_id=687ae3b37ca843aabd40a3218beb0092)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:49:08.372460 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5399e4c3-4c4b-416b-bc68-96e51df0050c admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:49:08.434198 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-5399e4c3-4c4b-416b-bc68-96e51df0050c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '687ae3b37ca843aabd40a3218beb0092', 'request_id': 'req-5399e4c3-4c4b-416b-bc68-96e51df0050c'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:49:08.442269 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-5399e4c3-4c4b-416b-bc68-96e51df0050c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '687ae3b37ca843aabd40a3218beb0092', 'request_id': 'req-5399e4c3-4c4b-416b-bc68-96e51df0050c'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:49:08.451843 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-5399e4c3-4c4b-416b-bc68-96e51df0050c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '687ae3b37ca843aabd40a3218beb0092', 'request_id': 'req-5399e4c3-4c4b-416b-bc68-96e51df0050c'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:49:08.464958 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-5399e4c3-4c4b-416b-bc68-96e51df0050c admin admin] The token cache is being invalidate because project 687ae3b37ca843aabd40a3218beb0092 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86828) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 10:49:08.465379 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-5399e4c3-4c4b-416b-bc68-96e51df0050c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:49:08.466144 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-5399e4c3-4c4b-416b-bc68-96e51df0050c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:49:08.470814 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1069/2167] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Jul 27 10:49:08 2022] DELETE /identity/v3/projects/687ae3b37ca843aabd40a3218beb0092 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:49:08.471769 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622c31ad-a276-4706-a05b-3ccebc95ea4a admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:49:08.471907 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5399e4c3-4c4b-416b-bc68-96e51df0050c admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:49:08.473202 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-5399e4c3-4c4b-416b-bc68-96e51df0050c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:49:08.480120 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:49:08.527429 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2874d8ef-26a3-44e7-aab3-7a885ae24977 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7922d0a1a7c4e808d0b2a80dca0da40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:49:08.528423 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2874d8ef-26a3-44e7-aab3-7a885ae24977 service neutron] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:49:08.528751 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2874d8ef-26a3-44e7-aab3-7a885ae24977 service neutron] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:49:08.529038 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2874d8ef-26a3-44e7-aab3-7a885ae24977 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:49:08.531356 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-2874d8ef-26a3-44e7-aab3-7a885ae24977 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:49:08.551257 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5399e4c3-4c4b-416b-bc68-96e51df0050c admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:49:08.557232 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6b6b7e-eef7-4ed3-a6d7-9601ab7da01d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:49:08.558664 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6b6b7e-eef7-4ed3-a6d7-9601ab7da01d admin admin] REQUEST_METHOD: `DELETE` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:49:08.559003 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6b6b7e-eef7-4ed3-a6d7-9601ab7da01d admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:49:08.559426 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6b6b7e-eef7-4ed3-a6d7-9601ab7da01d admin admin] PATH_INFO: `/v3/users/b6bc7f4437474795958c10e50c0d4e55` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:49:08.560741 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6b6b7e-eef7-4ed3-a6d7-9601ab7da01d admin admin] RBAC: Authorizing `identity:delete_user(user_id=b6bc7f4437474795958c10e50c0d4e55)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:49:08.562804 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6b6b7e-eef7-4ed3-a6d7-9601ab7da01d admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:49:08.601601 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2874d8ef-26a3-44e7-aab3-7a885ae24977 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:49:08.603079 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2874d8ef-26a3-44e7-aab3-7a885ae24977 service neutron] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:49:08.611742 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-7b6b6b7e-eef7-4ed3-a6d7-9601ab7da01d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b6bc7f4437474795958c10e50c0d4e55'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:49:08.613243 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1099/2168] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:49:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 10:49:08.618629 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-7b6b6b7e-eef7-4ed3-a6d7-9601ab7da01d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b6bc7f4437474795958c10e50c0d4e55'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:49:08.627008 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-7b6b6b7e-eef7-4ed3-a6d7-9601ab7da01d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b6bc7f4437474795958c10e50c0d4e55'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:49:08.627849 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-7b6b6b7e-eef7-4ed3-a6d7-9601ab7da01d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b6bc7f4437474795958c10e50c0d4e55'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:49:08.628505 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-7b6b6b7e-eef7-4ed3-a6d7-9601ab7da01d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b6bc7f4437474795958c10e50c0d4e55'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:49:08.646020 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1070/2169] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Jul 27 10:49:08 2022] DELETE /identity/v3/users/b6bc7f4437474795958c10e50c0d4e55 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 10:49:08.950293 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2874d8ef-26a3-44e7-aab3-7a885ae24977 service neutron] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:49:08.951941 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-2874d8ef-26a3-44e7-aab3-7a885ae24977 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:49:08.964561 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2874d8ef-26a3-44e7-aab3-7a885ae24977 service neutron] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:49:09.007623 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6b6b7e-eef7-4ed3-a6d7-9601ab7da01d admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:49:09.009317 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-7b6b6b7e-eef7-4ed3-a6d7-9601ab7da01d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:49:09.025048 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:49:09.040864 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46aeac3-c983-416d-9bf4-47414579e5e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:49:09.041517 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46aeac3-c983-416d-9bf4-47414579e5e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:49:09.041798 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46aeac3-c983-416d-9bf4-47414579e5e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:49:09.042227 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46aeac3-c983-416d-9bf4-47414579e5e9 admin admin] PATH_INFO: `/v3/projects/9d9fcadac6784925ae34dba862b9db9e` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:49:09.043893 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46aeac3-c983-416d-9bf4-47414579e5e9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9d9fcadac6784925ae34dba862b9db9e)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:49:09.045851 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46aeac3-c983-416d-9bf4-47414579e5e9 admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:49:09.088206 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6b6b7e-eef7-4ed3-a6d7-9601ab7da01d admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:49:09.097142 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-c46aeac3-c983-416d-9bf4-47414579e5e9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9d9fcadac6784925ae34dba862b9db9e', 'request_id': 'req-c46aeac3-c983-416d-9bf4-47414579e5e9'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:49:09.103747 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-c46aeac3-c983-416d-9bf4-47414579e5e9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9d9fcadac6784925ae34dba862b9db9e', 'request_id': 'req-c46aeac3-c983-416d-9bf4-47414579e5e9'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:49:09.113851 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-c46aeac3-c983-416d-9bf4-47414579e5e9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9d9fcadac6784925ae34dba862b9db9e', 'request_id': 'req-c46aeac3-c983-416d-9bf4-47414579e5e9'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:49:09.115998 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:49:09.125806 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-c46aeac3-c983-416d-9bf4-47414579e5e9 admin admin] The token cache is being invalidate because project 9d9fcadac6784925ae34dba862b9db9e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86827) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 10:49:09.126099 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-c46aeac3-c983-416d-9bf4-47414579e5e9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:49:09.127059 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-c46aeac3-c983-416d-9bf4-47414579e5e9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:49:09.128581 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1100/2170] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Jul 27 10:49:08 2022] DELETE /identity/v3/projects/9d9fcadac6784925ae34dba862b9db9e => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:49:09.132535 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46aeac3-c983-416d-9bf4-47414579e5e9 admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:49:09.133856 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-c46aeac3-c983-416d-9bf4-47414579e5e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:49:09.150099 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c99501b-5263-4d27-9fc0-68e73bb54dd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:49:09.150793 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c99501b-5263-4d27-9fc0-68e73bb54dd1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:49:09.151054 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c99501b-5263-4d27-9fc0-68e73bb54dd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:49:09.152025 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c99501b-5263-4d27-9fc0-68e73bb54dd1 admin admin] PATH_INFO: `/v3/projects/ecdaa8585aaa41a09c7938e3b30a6040` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:49:09.153116 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c99501b-5263-4d27-9fc0-68e73bb54dd1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ecdaa8585aaa41a09c7938e3b30a6040)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:49:09.155038 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c99501b-5263-4d27-9fc0-68e73bb54dd1 admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:49:09.202388 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-1c99501b-5263-4d27-9fc0-68e73bb54dd1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ecdaa8585aaa41a09c7938e3b30a6040', 'request_id': 'req-1c99501b-5263-4d27-9fc0-68e73bb54dd1'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:49:09.206848 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46aeac3-c983-416d-9bf4-47414579e5e9 admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:49:09.208504 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-1c99501b-5263-4d27-9fc0-68e73bb54dd1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ecdaa8585aaa41a09c7938e3b30a6040', 'request_id': 'req-1c99501b-5263-4d27-9fc0-68e73bb54dd1'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:49:09.217001 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-1c99501b-5263-4d27-9fc0-68e73bb54dd1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ecdaa8585aaa41a09c7938e3b30a6040', 'request_id': 'req-1c99501b-5263-4d27-9fc0-68e73bb54dd1'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:49:09.228165 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-1c99501b-5263-4d27-9fc0-68e73bb54dd1 admin admin] The token cache is being invalidate because project ecdaa8585aaa41a09c7938e3b30a6040 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86828) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 10:49:09.228457 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-1c99501b-5263-4d27-9fc0-68e73bb54dd1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:49:09.229086 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-1c99501b-5263-4d27-9fc0-68e73bb54dd1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:49:09.230869 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1071/2171] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Jul 27 10:49:09 2022] DELETE /identity/v3/projects/ecdaa8585aaa41a09c7938e3b30a6040 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 10:49:09.236354 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3cd5f1-7e68-412a-943c-8ec39a0879be None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:49:09.236639 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3cd5f1-7e68-412a-943c-8ec39a0879be None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:49:09.236912 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3cd5f1-7e68-412a-943c-8ec39a0879be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:49:09.256212 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-2d3cd5f1-7e68-412a-943c-8ec39a0879be None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:49:09.257414 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-2d3cd5f1-7e68-412a-943c-8ec39a0879be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:49:09.266707 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bed62f7-c5dc-4c4d-b303-98dd12523411 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:49:09.267364 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bed62f7-c5dc-4c4d-b303-98dd12523411 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:49:09.267649 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bed62f7-c5dc-4c4d-b303-98dd12523411 admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:49:09.267929 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bed62f7-c5dc-4c4d-b303-98dd12523411 admin admin] PATH_INFO: `/v3/users/881090625557466bb2b9c8fe01f1e4c2` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:49:09.269109 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bed62f7-c5dc-4c4d-b303-98dd12523411 admin admin] RBAC: Authorizing `identity:delete_user(user_id=881090625557466bb2b9c8fe01f1e4c2)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:49:09.271090 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bed62f7-c5dc-4c4d-b303-98dd12523411 admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:49:09.300575 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:49:09.319924 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1072/2172] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:49:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 10:49:09.327467 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-7bed62f7-c5dc-4c4d-b303-98dd12523411 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '881090625557466bb2b9c8fe01f1e4c2'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:49:09.328763 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-7bed62f7-c5dc-4c4d-b303-98dd12523411 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '881090625557466bb2b9c8fe01f1e4c2'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:49:09.328886 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6576fbe7-6cb0-42ba-8db9-f93fccfcb79e None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:49:09.329122 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6576fbe7-6cb0-42ba-8db9-f93fccfcb79e None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:49:09.329400 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6576fbe7-6cb0-42ba-8db9-f93fccfcb79e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:49:09.336264 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-7bed62f7-c5dc-4c4d-b303-98dd12523411 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '881090625557466bb2b9c8fe01f1e4c2'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:49:09.336912 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-7bed62f7-c5dc-4c4d-b303-98dd12523411 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '881090625557466bb2b9c8fe01f1e4c2'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:49:09.337735 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-7bed62f7-c5dc-4c4d-b303-98dd12523411 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '881090625557466bb2b9c8fe01f1e4c2'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:49:09.351216 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-6576fbe7-6cb0-42ba-8db9-f93fccfcb79e None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:49:09.352428 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-6576fbe7-6cb0-42ba-8db9-f93fccfcb79e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:49:09.357629 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1101/2173] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Jul 27 10:49:09 2022] DELETE /identity/v3/users/881090625557466bb2b9c8fe01f1e4c2 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 10:49:09.361061 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1073/2174] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:49:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 10:49:09.365041 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bed62f7-c5dc-4c4d-b303-98dd12523411 admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:49:09.432787 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bed62f7-c5dc-4c4d-b303-98dd12523411 admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:49:09.438063 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227e156b-ce63-4855-88ee-3f1e8e66c1e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:49:09.438727 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227e156b-ce63-4855-88ee-3f1e8e66c1e0 admin admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:49:09.439013 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227e156b-ce63-4855-88ee-3f1e8e66c1e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:49:09.439292 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227e156b-ce63-4855-88ee-3f1e8e66c1e0 admin admin] PATH_INFO: `/v3/domains` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:49:09.440011 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227e156b-ce63-4855-88ee-3f1e8e66c1e0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86827) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:49:09.440317 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227e156b-ce63-4855-88ee-3f1e8e66c1e0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:49:09.442069 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227e156b-ce63-4855-88ee-3f1e8e66c1e0 admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:49:09.460858 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1102/2175] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Jul 27 10:49:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:49:09.466025 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6576fbe7-6cb0-42ba-8db9-f93fccfcb79e None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:49:09.477962 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6576fbe7-6cb0-42ba-8db9-f93fccfcb79e None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:49:09.483950 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd72b9f-26eb-4ef6-8867-51a4adfd19bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:49:09.484697 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd72b9f-26eb-4ef6-8867-51a4adfd19bb admin admin] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:49:09.485171 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd72b9f-26eb-4ef6-8867-51a4adfd19bb admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:49:09.485599 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd72b9f-26eb-4ef6-8867-51a4adfd19bb admin admin] PATH_INFO: `/v3/projects` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:49:09.486796 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd72b9f-26eb-4ef6-8867-51a4adfd19bb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:49:09.488556 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd72b9f-26eb-4ef6-8867-51a4adfd19bb admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:49:09.519986 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1074/2176] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Jul 27 10:49:09 2022] POST /identity/v3/projects => generated 371 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 10:49:09.524342 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227e156b-ce63-4855-88ee-3f1e8e66c1e0 admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:49:09.535398 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227e156b-ce63-4855-88ee-3f1e8e66c1e0 admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:49:09.599659 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843eeb97-33cc-417d-a134-5e6e1039eefb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:49:09.600358 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843eeb97-33cc-417d-a134-5e6e1039eefb admin admin] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:49:09.600712 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843eeb97-33cc-417d-a134-5e6e1039eefb admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:49:09.601060 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843eeb97-33cc-417d-a134-5e6e1039eefb admin admin] PATH_INFO: `/v3/users` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:49:09.601805 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843eeb97-33cc-417d-a134-5e6e1039eefb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:49:09.603873 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843eeb97-33cc-417d-a134-5e6e1039eefb admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:49:09.637629 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1103/2177] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Jul 27 10:49:09 2022] POST /identity/v3/users => generated 286 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 10:49:09.641510 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd72b9f-26eb-4ef6-8867-51a4adfd19bb admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:49:09.652028 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd72b9f-26eb-4ef6-8867-51a4adfd19bb admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:49:09.656630 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355ba347-4a6e-4619-8999-3362afe933a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:49:09.657249 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355ba347-4a6e-4619-8999-3362afe933a7 admin admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:49:09.657529 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355ba347-4a6e-4619-8999-3362afe933a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:49:09.657805 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355ba347-4a6e-4619-8999-3362afe933a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:49:09.658491 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355ba347-4a6e-4619-8999-3362afe933a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:49:09.660132 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355ba347-4a6e-4619-8999-3362afe933a7 admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:49:09.672158 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1075/2178] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 10:49:09 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 10:49:09.676259 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843eeb97-33cc-417d-a134-5e6e1039eefb admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:49:09.686447 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843eeb97-33cc-417d-a134-5e6e1039eefb admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:49:09.691477 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de92546-981f-4500-9327-a98a65f88312 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:49:09.692102 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de92546-981f-4500-9327-a98a65f88312 admin admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:49:09.692388 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de92546-981f-4500-9327-a98a65f88312 admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:49:09.692668 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de92546-981f-4500-9327-a98a65f88312 admin admin] PATH_INFO: `/v3/roles` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:49:09.693370 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de92546-981f-4500-9327-a98a65f88312 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:49:09.695022 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de92546-981f-4500-9327-a98a65f88312 admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:49:09.706270 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1104/2179] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 10:49:09 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 10:49:09.710536 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355ba347-4a6e-4619-8999-3362afe933a7 admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:49:09.721480 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355ba347-4a6e-4619-8999-3362afe933a7 admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:49:09.727446 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae4e737-b067-4ec6-bbcc-2781328ca4ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:49:09.728134 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae4e737-b067-4ec6-bbcc-2781328ca4ea admin admin] REQUEST_METHOD: `PUT` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:49:09.728472 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae4e737-b067-4ec6-bbcc-2781328ca4ea admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:49:09.728726 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae4e737-b067-4ec6-bbcc-2781328ca4ea admin admin] PATH_INFO: `/v3/projects/a99c3d709a5a475aa49411fd80e616e5/users/dc28f7f7de764e91ade03830248a4e21/roles/4c9fc0b3baed441da2126be41cc6b04a` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:49:09.746172 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae4e737-b067-4ec6-bbcc-2781328ca4ea admin admin] RBAC: Authorizing `identity:create_grant(project_id=a99c3d709a5a475aa49411fd80e616e5, user_id=dc28f7f7de764e91ade03830248a4e21, role_id=4c9fc0b3baed441da2126be41cc6b04a)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:49:09.748242 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de92546-981f-4500-9327-a98a65f88312 admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:49:09.750053 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-8de92546-981f-4500-9327-a98a65f88312 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:49:09.750896 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae4e737-b067-4ec6-bbcc-2781328ca4ea admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:49:09.754546 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:49:09.767139 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de92546-981f-4500-9327-a98a65f88312 admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:49:09.768481 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1076/2180] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 10:49:09 2022] PUT /identity/v3/projects/a99c3d709a5a475aa49411fd80e616e5/users/dc28f7f7de764e91ade03830248a4e21/roles/4c9fc0b3baed441da2126be41cc6b04a => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 10:49:09.781474 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861b7916-cbec-4bd5-ad9e-cd82ac043a8e None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:49:09.783065 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861b7916-cbec-4bd5-ad9e-cd82ac043a8e None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:49:09.786521 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861b7916-cbec-4bd5-ad9e-cd82ac043a8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:49:09.791952 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:49:09.830207 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-861b7916-cbec-4bd5-ad9e-cd82ac043a8e None None] MFA Rules not processed for user `dc28f7f7de764e91ade03830248a4e21`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:49:09.831620 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-861b7916-cbec-4bd5-ad9e-cd82ac043a8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:49:09.855708 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386cf312-f5be-4284-b8df-145bb37fa3bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:49:09.856382 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386cf312-f5be-4284-b8df-145bb37fa3bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:49:09.856666 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386cf312-f5be-4284-b8df-145bb37fa3bb admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:49:09.856944 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386cf312-f5be-4284-b8df-145bb37fa3bb admin admin] PATH_INFO: `/v3/projects/5303a1c35c634f8aafc353bcea56e9d6` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:49:09.858567 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386cf312-f5be-4284-b8df-145bb37fa3bb admin admin] RBAC: Authorizing `identity:delete_project(project_id=5303a1c35c634f8aafc353bcea56e9d6)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:49:09.861490 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386cf312-f5be-4284-b8df-145bb37fa3bb admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:49:09.913512 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1077/2181] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:49:09 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 10:49:09.923236 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-386cf312-f5be-4284-b8df-145bb37fa3bb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5303a1c35c634f8aafc353bcea56e9d6', 'request_id': 'req-386cf312-f5be-4284-b8df-145bb37fa3bb'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:49:09.926403 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861b7916-cbec-4bd5-ad9e-cd82ac043a8e None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:49:09.927999 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-861b7916-cbec-4bd5-ad9e-cd82ac043a8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:49:09.929908 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-386cf312-f5be-4284-b8df-145bb37fa3bb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5303a1c35c634f8aafc353bcea56e9d6', 'request_id': 'req-386cf312-f5be-4284-b8df-145bb37fa3bb'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:49:09.937666 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-386cf312-f5be-4284-b8df-145bb37fa3bb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5303a1c35c634f8aafc353bcea56e9d6', 'request_id': 'req-386cf312-f5be-4284-b8df-145bb37fa3bb'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:49:09.952105 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-386cf312-f5be-4284-b8df-145bb37fa3bb admin admin] The token cache is being invalidate because project 5303a1c35c634f8aafc353bcea56e9d6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86827) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 10:49:09.952703 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-386cf312-f5be-4284-b8df-145bb37fa3bb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:49:09.953140 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-386cf312-f5be-4284-b8df-145bb37fa3bb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:49:09.954796 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1105/2182] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Jul 27 10:49:09 2022] DELETE /identity/v3/projects/5303a1c35c634f8aafc353bcea56e9d6 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 10:49:09.958508 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386cf312-f5be-4284-b8df-145bb37fa3bb admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:49:09.959937 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-386cf312-f5be-4284-b8df-145bb37fa3bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:49:10.009081 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861b7916-cbec-4bd5-ad9e-cd82ac043a8e None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:49:10.024616 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386cf312-f5be-4284-b8df-145bb37fa3bb admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:49:10.030019 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c4ac12-06f4-4bd3-ba5b-b0379300ce40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:49:10.030718 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c4ac12-06f4-4bd3-ba5b-b0379300ce40 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:49:10.031003 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c4ac12-06f4-4bd3-ba5b-b0379300ce40 admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:49:10.031280 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c4ac12-06f4-4bd3-ba5b-b0379300ce40 admin admin] PATH_INFO: `/v3/users/a98a07c7b7da4c8bbbdb045245d62b72` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:49:10.032468 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c4ac12-06f4-4bd3-ba5b-b0379300ce40 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a98a07c7b7da4c8bbbdb045245d62b72)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:49:10.034393 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c4ac12-06f4-4bd3-ba5b-b0379300ce40 admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:49:10.061629 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410c353f-ec87-4c9b-8419-68d93f9f3e24 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc03c5d232654878aa1ddce15f49fdaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:49:10.062201 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410c353f-ec87-4c9b-8419-68d93f9f3e24 service glance] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:49:10.062703 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410c353f-ec87-4c9b-8419-68d93f9f3e24 service glance] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:49:10.062972 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410c353f-ec87-4c9b-8419-68d93f9f3e24 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:49:10.064862 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-410c353f-ec87-4c9b-8419-68d93f9f3e24 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:49:10.082619 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-71c4ac12-06f4-4bd3-ba5b-b0379300ce40 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a98a07c7b7da4c8bbbdb045245d62b72'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:49:10.084658 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:49:10.097459 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-71c4ac12-06f4-4bd3-ba5b-b0379300ce40 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a98a07c7b7da4c8bbbdb045245d62b72'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:49:10.109074 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-71c4ac12-06f4-4bd3-ba5b-b0379300ce40 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a98a07c7b7da4c8bbbdb045245d62b72'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:49:10.110114 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-71c4ac12-06f4-4bd3-ba5b-b0379300ce40 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a98a07c7b7da4c8bbbdb045245d62b72'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:49:10.111067 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-71c4ac12-06f4-4bd3-ba5b-b0379300ce40 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a98a07c7b7da4c8bbbdb045245d62b72'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:49:10.114892 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:49:10.132943 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1106/2183] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Jul 27 10:49:09 2022] DELETE /identity/v3/users/a98a07c7b7da4c8bbbdb045245d62b72 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 10:49:10.144569 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410c353f-ec87-4c9b-8419-68d93f9f3e24 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:49:10.146167 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410c353f-ec87-4c9b-8419-68d93f9f3e24 service glance] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:49:10.155908 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1078/2184] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:49:09 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 10:49:10.424333 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c4ac12-06f4-4bd3-ba5b-b0379300ce40 admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:49:10.426635 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-71c4ac12-06f4-4bd3-ba5b-b0379300ce40 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:49:10.509106 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c4ac12-06f4-4bd3-ba5b-b0379300ce40 admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:49:10.514108 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708d8fad-5509-4064-9b5c-8c5e00544f10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:49:10.514791 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708d8fad-5509-4064-9b5c-8c5e00544f10 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:49:10.515077 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708d8fad-5509-4064-9b5c-8c5e00544f10 admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:49:10.515352 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708d8fad-5509-4064-9b5c-8c5e00544f10 admin admin] PATH_INFO: `/v3/projects/cd226bf11d0c4401bdb59b79b302cafe` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:49:10.516526 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708d8fad-5509-4064-9b5c-8c5e00544f10 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cd226bf11d0c4401bdb59b79b302cafe)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:49:10.518464 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708d8fad-5509-4064-9b5c-8c5e00544f10 admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:49:10.582417 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-708d8fad-5509-4064-9b5c-8c5e00544f10 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cd226bf11d0c4401bdb59b79b302cafe', 'request_id': 'req-708d8fad-5509-4064-9b5c-8c5e00544f10'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:49:10.590091 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-708d8fad-5509-4064-9b5c-8c5e00544f10 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cd226bf11d0c4401bdb59b79b302cafe', 'request_id': 'req-708d8fad-5509-4064-9b5c-8c5e00544f10'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:49:10.609563 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-708d8fad-5509-4064-9b5c-8c5e00544f10 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cd226bf11d0c4401bdb59b79b302cafe', 'request_id': 'req-708d8fad-5509-4064-9b5c-8c5e00544f10'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:49:10.632675 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-708d8fad-5509-4064-9b5c-8c5e00544f10 admin admin] The token cache is being invalidate because project cd226bf11d0c4401bdb59b79b302cafe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86827) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 10:49:10.632986 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-708d8fad-5509-4064-9b5c-8c5e00544f10 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:49:10.633795 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-708d8fad-5509-4064-9b5c-8c5e00544f10 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:49:10.635499 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1107/2185] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Jul 27 10:49:10 2022] DELETE /identity/v3/projects/cd226bf11d0c4401bdb59b79b302cafe => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:49:10.640388 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959dcaf4-eca1-4aa6-a6b4-ffbc43c176c4 None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:49:10.640867 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959dcaf4-eca1-4aa6-a6b4-ffbc43c176c4 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:49:10.640961 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959dcaf4-eca1-4aa6-a6b4-ffbc43c176c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:49:10.694846 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-959dcaf4-eca1-4aa6-a6b4-ffbc43c176c4 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:49:10.696075 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-959dcaf4-eca1-4aa6-a6b4-ffbc43c176c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:49:10.763094 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1079/2186] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:49:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:49:10.770579 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd09673-80c3-45f1-a4a8-aa353968906f None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:49:10.770898 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd09673-80c3-45f1-a4a8-aa353968906f None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:49:10.771178 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd09673-80c3-45f1-a4a8-aa353968906f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:49:10.791456 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-6fd09673-80c3-45f1-a4a8-aa353968906f None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:49:10.792661 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-6fd09673-80c3-45f1-a4a8-aa353968906f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:49:10.800911 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1108/2187] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:49:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:49:10.804959 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959dcaf4-eca1-4aa6-a6b4-ffbc43c176c4 None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:49:10.814558 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959dcaf4-eca1-4aa6-a6b4-ffbc43c176c4 None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:49:10.819536 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d18ab6-66d5-43c8-a6a9-c6c175faf6fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:49:10.820273 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d18ab6-66d5-43c8-a6a9-c6c175faf6fb admin admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:49:10.820550 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d18ab6-66d5-43c8-a6a9-c6c175faf6fb admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:49:10.820819 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d18ab6-66d5-43c8-a6a9-c6c175faf6fb admin admin] PATH_INFO: `/v3/domains` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:49:10.821712 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d18ab6-66d5-43c8-a6a9-c6c175faf6fb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86828) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:49:10.822184 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d18ab6-66d5-43c8-a6a9-c6c175faf6fb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:49:10.824431 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d18ab6-66d5-43c8-a6a9-c6c175faf6fb admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:49:10.841149 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1080/2188] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Jul 27 10:49:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:49:10.842181 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd09673-80c3-45f1-a4a8-aa353968906f None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:49:10.851595 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd09673-80c3-45f1-a4a8-aa353968906f None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:49:10.856562 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62529a6b-200a-4b41-8e5c-d81e2b52bc48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:49:10.857181 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62529a6b-200a-4b41-8e5c-d81e2b52bc48 admin admin] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:49:10.857464 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62529a6b-200a-4b41-8e5c-d81e2b52bc48 admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:49:10.857740 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62529a6b-200a-4b41-8e5c-d81e2b52bc48 admin admin] PATH_INFO: `/v3/projects` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:49:10.858419 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62529a6b-200a-4b41-8e5c-d81e2b52bc48 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:49:10.860443 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62529a6b-200a-4b41-8e5c-d81e2b52bc48 admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:49:10.895240 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1109/2189] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Jul 27 10:49:10 2022] POST /identity/v3/projects => generated 373 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 10:49:10.899337 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d18ab6-66d5-43c8-a6a9-c6c175faf6fb admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:49:10.912099 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d18ab6-66d5-43c8-a6a9-c6c175faf6fb admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:49:10.971319 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69827048-3503-4093-97da-9a3083af0424 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:49:10.971950 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69827048-3503-4093-97da-9a3083af0424 admin admin] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:49:10.972237 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69827048-3503-4093-97da-9a3083af0424 admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:49:10.972521 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69827048-3503-4093-97da-9a3083af0424 admin admin] PATH_INFO: `/v3/users` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:49:10.973199 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69827048-3503-4093-97da-9a3083af0424 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:49:10.975160 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69827048-3503-4093-97da-9a3083af0424 admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:49:10.996167 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1081/2190] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Jul 27 10:49:10 2022] POST /identity/v3/users => generated 287 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 10:49:11.000356 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62529a6b-200a-4b41-8e5c-d81e2b52bc48 admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:49:11.011433 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62529a6b-200a-4b41-8e5c-d81e2b52bc48 admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:49:11.016824 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd24471-93fa-46c4-92ec-7b9404c94f27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:49:11.017438 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd24471-93fa-46c4-92ec-7b9404c94f27 admin admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:49:11.017717 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd24471-93fa-46c4-92ec-7b9404c94f27 admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:49:11.017992 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd24471-93fa-46c4-92ec-7b9404c94f27 admin admin] PATH_INFO: `/v3/roles` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:49:11.018712 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd24471-93fa-46c4-92ec-7b9404c94f27 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:49:11.020290 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd24471-93fa-46c4-92ec-7b9404c94f27 admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:49:11.031091 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1110/2191] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 10:49:10 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 10:49:11.035159 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69827048-3503-4093-97da-9a3083af0424 admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:49:11.045250 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69827048-3503-4093-97da-9a3083af0424 admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:49:11.049971 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfecd7b-8333-44a4-b08c-5f006f7c238d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:49:11.050590 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfecd7b-8333-44a4-b08c-5f006f7c238d admin admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:49:11.050906 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfecd7b-8333-44a4-b08c-5f006f7c238d admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:49:11.051187 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfecd7b-8333-44a4-b08c-5f006f7c238d admin admin] PATH_INFO: `/v3/roles` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:49:11.051915 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfecd7b-8333-44a4-b08c-5f006f7c238d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:49:11.053515 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfecd7b-8333-44a4-b08c-5f006f7c238d admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:49:11.064611 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1082/2192] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 10:49:11 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 10:49:11.068655 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd24471-93fa-46c4-92ec-7b9404c94f27 admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:49:11.079048 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd24471-93fa-46c4-92ec-7b9404c94f27 admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:49:11.083579 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fcfcf6-bc21-4b5a-a988-a948a6805183 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:49:11.084235 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fcfcf6-bc21-4b5a-a988-a948a6805183 admin admin] REQUEST_METHOD: `PUT` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:49:11.084514 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fcfcf6-bc21-4b5a-a988-a948a6805183 admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:49:11.084789 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fcfcf6-bc21-4b5a-a988-a948a6805183 admin admin] PATH_INFO: `/v3/projects/543367ebb0cc4760a1cad402d79e8fb5/users/ef68584396384a7c9ee9962914a171da/roles/4c9fc0b3baed441da2126be41cc6b04a` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:49:11.099628 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fcfcf6-bc21-4b5a-a988-a948a6805183 admin admin] RBAC: Authorizing `identity:create_grant(project_id=543367ebb0cc4760a1cad402d79e8fb5, user_id=ef68584396384a7c9ee9962914a171da, role_id=4c9fc0b3baed441da2126be41cc6b04a)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:49:11.102282 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fcfcf6-bc21-4b5a-a988-a948a6805183 admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:49:11.104545 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:49:11.117439 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1111/2193] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 10:49:11 2022] PUT /identity/v3/projects/543367ebb0cc4760a1cad402d79e8fb5/users/ef68584396384a7c9ee9962914a171da/roles/4c9fc0b3baed441da2126be41cc6b04a => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:49:11.129553 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc186c97-17c7-4f38-ada7-4d18060254ab None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:49:11.129842 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc186c97-17c7-4f38-ada7-4d18060254ab None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:49:11.130117 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc186c97-17c7-4f38-ada7-4d18060254ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:49:11.157377 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-bc186c97-17c7-4f38-ada7-4d18060254ab None None] MFA Rules not processed for user `ef68584396384a7c9ee9962914a171da`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:49:11.158779 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-bc186c97-17c7-4f38-ada7-4d18060254ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:49:11.189676 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:49:11.231460 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1083/2194] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:49:11 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:49:11.242257 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fcfcf6-bc21-4b5a-a988-a948a6805183 admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:49:11.243963 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-a6fcfcf6-bc21-4b5a-a988-a948a6805183 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:49:11.306068 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fcfcf6-bc21-4b5a-a988-a948a6805183 admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:49:11.311016 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153f0f1b-b5e8-4e73-ae1b-ff4e100738dc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '864d35f268cf45148756109759bb2743', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:49:11.311656 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153f0f1b-b5e8-4e73-ae1b-ff4e100738dc service cinder] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:49:11.311941 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153f0f1b-b5e8-4e73-ae1b-ff4e100738dc service cinder] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:49:11.312353 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153f0f1b-b5e8-4e73-ae1b-ff4e100738dc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:49:11.321499 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153f0f1b-b5e8-4e73-ae1b-ff4e100738dc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:49:11.323056 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153f0f1b-b5e8-4e73-ae1b-ff4e100738dc service cinder] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:49:11.335256 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1112/2195] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:49:11 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 10:49:12.740403 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81a812a-54c0-4e39-977b-8dc0791171df None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:49:12.741062 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81a812a-54c0-4e39-977b-8dc0791171df None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:49:12.741062 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81a812a-54c0-4e39-977b-8dc0791171df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:49:12.760912 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-a81a812a-54c0-4e39-977b-8dc0791171df None None] MFA Rules not processed for user `ef68584396384a7c9ee9962914a171da`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:49:12.762498 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-a81a812a-54c0-4e39-977b-8dc0791171df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:49:12.769585 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1084/2196] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:49:12 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:49:16.553808 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153f0f1b-b5e8-4e73-ae1b-ff4e100738dc service cinder] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:49:16.555427 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-153f0f1b-b5e8-4e73-ae1b-ff4e100738dc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:49:16.738959 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153f0f1b-b5e8-4e73-ae1b-ff4e100738dc service cinder] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:49:16.752117 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0da5e67-852c-4745-8ab3-a537e1b52411 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:49:16.752951 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0da5e67-852c-4745-8ab3-a537e1b52411 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:49:16.753240 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0da5e67-852c-4745-8ab3-a537e1b52411 admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:49:16.753520 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0da5e67-852c-4745-8ab3-a537e1b52411 admin admin] PATH_INFO: `/v3/users/19317a4ce6fe430abc4cf2d10a0f5772` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:49:16.755020 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0da5e67-852c-4745-8ab3-a537e1b52411 admin admin] RBAC: Authorizing `identity:delete_user(user_id=19317a4ce6fe430abc4cf2d10a0f5772)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:49:16.761082 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0da5e67-852c-4745-8ab3-a537e1b52411 admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:49:16.878082 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-b0da5e67-852c-4745-8ab3-a537e1b52411 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '19317a4ce6fe430abc4cf2d10a0f5772'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:49:16.897650 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-b0da5e67-852c-4745-8ab3-a537e1b52411 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '19317a4ce6fe430abc4cf2d10a0f5772'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:49:16.935380 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-b0da5e67-852c-4745-8ab3-a537e1b52411 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '19317a4ce6fe430abc4cf2d10a0f5772'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:49:16.937627 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-b0da5e67-852c-4745-8ab3-a537e1b52411 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '19317a4ce6fe430abc4cf2d10a0f5772'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:49:16.939923 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-b0da5e67-852c-4745-8ab3-a537e1b52411 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '19317a4ce6fe430abc4cf2d10a0f5772'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:49:16.984420 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1113/2197] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Jul 27 10:49:16 2022] DELETE /identity/v3/users/19317a4ce6fe430abc4cf2d10a0f5772 => generated 0 bytes in 431 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:49:16.989910 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3af7b15-4493-45ad-bdad-bca09b309985 None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:49:16.990530 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3af7b15-4493-45ad-bdad-bca09b309985 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:49:16.991534 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3af7b15-4493-45ad-bdad-bca09b309985 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:49:17.020405 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-e3af7b15-4493-45ad-bdad-bca09b309985 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:49:17.021914 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-e3af7b15-4493-45ad-bdad-bca09b309985 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:49:17.102662 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1085/2198] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:49:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:49:17.133436 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0da5e67-852c-4745-8ab3-a537e1b52411 admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:49:17.135090 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-b0da5e67-852c-4745-8ab3-a537e1b52411 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:49:17.165463 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:INSERT=2,keystone:SELECT=88,keystone:DELETE=10 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:49:17.283154 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0da5e67-852c-4745-8ab3-a537e1b52411 admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:49:17.287177 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb0c7ea-9751-43e1-9d1d-8ea674893226 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7922d0a1a7c4e808d0b2a80dca0da40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:49:17.288581 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb0c7ea-9751-43e1-9d1d-8ea674893226 service neutron] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:49:17.288581 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb0c7ea-9751-43e1-9d1d-8ea674893226 service neutron] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:49:17.288581 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb0c7ea-9751-43e1-9d1d-8ea674893226 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:49:17.299893 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb0c7ea-9751-43e1-9d1d-8ea674893226 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:49:17.301635 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb0c7ea-9751-43e1-9d1d-8ea674893226 service neutron] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:49:17.326804 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1114/2199] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:49:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 10:49:18.195435 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3af7b15-4493-45ad-bdad-bca09b309985 None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:49:18.196919 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-e3af7b15-4493-45ad-bdad-bca09b309985 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:49:18.209615 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3af7b15-4493-45ad-bdad-bca09b309985 None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:49:18.215775 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4652cf83-2eb8-419f-8213-0d018aa213a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:49:18.216681 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4652cf83-2eb8-419f-8213-0d018aa213a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:49:18.217124 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4652cf83-2eb8-419f-8213-0d018aa213a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:49:18.217559 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4652cf83-2eb8-419f-8213-0d018aa213a7 admin admin] PATH_INFO: `/v3/users/d80b5a817afd48ec9fb87b63b8d85e9a` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:49:18.218968 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4652cf83-2eb8-419f-8213-0d018aa213a7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d80b5a817afd48ec9fb87b63b8d85e9a)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:49:18.221301 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4652cf83-2eb8-419f-8213-0d018aa213a7 admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:49:18.304008 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-4652cf83-2eb8-419f-8213-0d018aa213a7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd80b5a817afd48ec9fb87b63b8d85e9a'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:49:18.312375 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-4652cf83-2eb8-419f-8213-0d018aa213a7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd80b5a817afd48ec9fb87b63b8d85e9a'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:49:18.325116 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-4652cf83-2eb8-419f-8213-0d018aa213a7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd80b5a817afd48ec9fb87b63b8d85e9a'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:49:18.325959 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-4652cf83-2eb8-419f-8213-0d018aa213a7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd80b5a817afd48ec9fb87b63b8d85e9a'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:49:18.327172 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-4652cf83-2eb8-419f-8213-0d018aa213a7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd80b5a817afd48ec9fb87b63b8d85e9a'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:49:18.360731 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb0c7ea-9751-43e1-9d1d-8ea674893226 service neutron] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:49:18.363342 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-ffb0c7ea-9751-43e1-9d1d-8ea674893226 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:49:18.367147 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1086/2200] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Jul 27 10:49:18 2022] DELETE /identity/v3/users/d80b5a817afd48ec9fb87b63b8d85e9a => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:49:18.369052 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b406793-d02f-4a94-8365-fb1f7bb17bd8 None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:49:18.369360 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b406793-d02f-4a94-8365-fb1f7bb17bd8 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:49:18.369645 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b406793-d02f-4a94-8365-fb1f7bb17bd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:49:18.408570 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-4b406793-d02f-4a94-8365-fb1f7bb17bd8 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:49:18.409979 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-4b406793-d02f-4a94-8365-fb1f7bb17bd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:49:18.411980 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb0c7ea-9751-43e1-9d1d-8ea674893226 service neutron] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:49:18.484805 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:49:18.512254 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf1002c-3cfc-4e57-bce1-e77fd8cee4f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:49:18.513238 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf1002c-3cfc-4e57-bce1-e77fd8cee4f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:49:18.513635 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf1002c-3cfc-4e57-bce1-e77fd8cee4f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:49:18.514019 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf1002c-3cfc-4e57-bce1-e77fd8cee4f5 admin admin] PATH_INFO: `/v3/projects/82101cb97faa4613a27f33a4871b7d9f` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:49:18.515374 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf1002c-3cfc-4e57-bce1-e77fd8cee4f5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=82101cb97faa4613a27f33a4871b7d9f)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:49:18.517817 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf1002c-3cfc-4e57-bce1-e77fd8cee4f5 admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:49:18.577582 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1087/2201] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:49:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:49:18.586691 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-6cf1002c-3cfc-4e57-bce1-e77fd8cee4f5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '82101cb97faa4613a27f33a4871b7d9f', 'request_id': 'req-6cf1002c-3cfc-4e57-bce1-e77fd8cee4f5'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:49:18.595659 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-6cf1002c-3cfc-4e57-bce1-e77fd8cee4f5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '82101cb97faa4613a27f33a4871b7d9f', 'request_id': 'req-6cf1002c-3cfc-4e57-bce1-e77fd8cee4f5'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:49:18.606159 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-6cf1002c-3cfc-4e57-bce1-e77fd8cee4f5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '82101cb97faa4613a27f33a4871b7d9f', 'request_id': 'req-6cf1002c-3cfc-4e57-bce1-e77fd8cee4f5'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:49:18.620989 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-6cf1002c-3cfc-4e57-bce1-e77fd8cee4f5 admin admin] The token cache is being invalidate because project 82101cb97faa4613a27f33a4871b7d9f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86827) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 10:49:18.621288 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-6cf1002c-3cfc-4e57-bce1-e77fd8cee4f5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:49:18.622104 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-6cf1002c-3cfc-4e57-bce1-e77fd8cee4f5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:49:18.622732 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b406793-d02f-4a94-8365-fb1f7bb17bd8 None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:49:18.624449 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-4b406793-d02f-4a94-8365-fb1f7bb17bd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:49:18.636528 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1115/2202] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Jul 27 10:49:18 2022] DELETE /identity/v3/projects/82101cb97faa4613a27f33a4871b7d9f => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:49:18.855548 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b406793-d02f-4a94-8365-fb1f7bb17bd8 None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:49:19.001167 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d32a145-8e74-4b73-9bec-5fb154874f17 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7922d0a1a7c4e808d0b2a80dca0da40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:49:19.004580 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d32a145-8e74-4b73-9bec-5fb154874f17 service neutron] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:49:19.005485 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d32a145-8e74-4b73-9bec-5fb154874f17 service neutron] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:49:19.006063 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d32a145-8e74-4b73-9bec-5fb154874f17 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:49:19.014040 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-6d32a145-8e74-4b73-9bec-5fb154874f17 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:49:19.175777 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d32a145-8e74-4b73-9bec-5fb154874f17 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:49:19.177571 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d32a145-8e74-4b73-9bec-5fb154874f17 service neutron] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:49:19.192298 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1088/2203] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:49:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 570 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 10:49:19.830312 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf1002c-3cfc-4e57-bce1-e77fd8cee4f5 admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:49:19.831801 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-6cf1002c-3cfc-4e57-bce1-e77fd8cee4f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:49:19.847539 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf1002c-3cfc-4e57-bce1-e77fd8cee4f5 admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:49:19.852355 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1dcd23-186a-4572-a2ae-269fc3bb5b09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:49:19.853578 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1dcd23-186a-4572-a2ae-269fc3bb5b09 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:49:19.853578 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1dcd23-186a-4572-a2ae-269fc3bb5b09 admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:49:19.853768 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1dcd23-186a-4572-a2ae-269fc3bb5b09 admin admin] PATH_INFO: `/v3/users/516a5f83148d4ccb885b3a901f14f8fa` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:49:19.854967 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1dcd23-186a-4572-a2ae-269fc3bb5b09 admin admin] RBAC: Authorizing `identity:delete_user(user_id=516a5f83148d4ccb885b3a901f14f8fa)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:49:19.856910 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1dcd23-186a-4572-a2ae-269fc3bb5b09 admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:49:19.907144 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d32a145-8e74-4b73-9bec-5fb154874f17 service neutron] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:49:19.911911 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-6d32a145-8e74-4b73-9bec-5fb154874f17 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:49:19.928526 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d32a145-8e74-4b73-9bec-5fb154874f17 service neutron] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:49:19.930014 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:49:19.939020 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b807e2-3fd2-4845-81f2-3aeb7d04bf25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:49:19.940264 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b807e2-3fd2-4845-81f2-3aeb7d04bf25 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:49:19.940732 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-9e1dcd23-186a-4572-a2ae-269fc3bb5b09 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '516a5f83148d4ccb885b3a901f14f8fa'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:49:19.941072 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b807e2-3fd2-4845-81f2-3aeb7d04bf25 admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:49:19.941572 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b807e2-3fd2-4845-81f2-3aeb7d04bf25 admin admin] PATH_INFO: `/v3/projects/3df1b61ddbb849aaaac33e3b04ffca32` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:49:19.947086 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b807e2-3fd2-4845-81f2-3aeb7d04bf25 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3df1b61ddbb849aaaac33e3b04ffca32)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:49:19.950586 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b807e2-3fd2-4845-81f2-3aeb7d04bf25 admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:49:19.952093 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-9e1dcd23-186a-4572-a2ae-269fc3bb5b09 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '516a5f83148d4ccb885b3a901f14f8fa'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:49:19.962764 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-9e1dcd23-186a-4572-a2ae-269fc3bb5b09 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '516a5f83148d4ccb885b3a901f14f8fa'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:49:19.963674 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-9e1dcd23-186a-4572-a2ae-269fc3bb5b09 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '516a5f83148d4ccb885b3a901f14f8fa'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:49:19.965505 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-9e1dcd23-186a-4572-a2ae-269fc3bb5b09 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '516a5f83148d4ccb885b3a901f14f8fa'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:49:19.991966 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1116/2204] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Jul 27 10:49:19 2022] DELETE /identity/v3/users/516a5f83148d4ccb885b3a901f14f8fa => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:49:19.996291 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b02ab17-720f-4330-b016-81663d617834 None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:49:19.997213 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b02ab17-720f-4330-b016-81663d617834 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:49:19.997793 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b02ab17-720f-4330-b016-81663d617834 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:49:20.001885 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:49:20.015322 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-05b807e2-3fd2-4845-81f2-3aeb7d04bf25 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3df1b61ddbb849aaaac33e3b04ffca32', 'request_id': 'req-05b807e2-3fd2-4845-81f2-3aeb7d04bf25'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:49:20.021990 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-05b807e2-3fd2-4845-81f2-3aeb7d04bf25 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3df1b61ddbb849aaaac33e3b04ffca32', 'request_id': 'req-05b807e2-3fd2-4845-81f2-3aeb7d04bf25'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:49:20.030439 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-2b02ab17-720f-4330-b016-81663d617834 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:49:20.031972 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-2b02ab17-720f-4330-b016-81663d617834 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:49:20.035089 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-05b807e2-3fd2-4845-81f2-3aeb7d04bf25 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3df1b61ddbb849aaaac33e3b04ffca32', 'request_id': 'req-05b807e2-3fd2-4845-81f2-3aeb7d04bf25'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:49:20.063120 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-05b807e2-3fd2-4845-81f2-3aeb7d04bf25 admin admin] The token cache is being invalidate because project 3df1b61ddbb849aaaac33e3b04ffca32 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86828) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 10:49:20.063426 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-05b807e2-3fd2-4845-81f2-3aeb7d04bf25 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:49:20.064124 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-05b807e2-3fd2-4845-81f2-3aeb7d04bf25 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:49:20.066046 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1089/2205] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Jul 27 10:49:19 2022] DELETE /identity/v3/projects/3df1b61ddbb849aaaac33e3b04ffca32 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 10:49:20.076465 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcda4be8-ce05-49b2-823f-c6af8aab485d None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:49:20.076905 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcda4be8-ce05-49b2-823f-c6af8aab485d None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:49:20.077287 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcda4be8-ce05-49b2-823f-c6af8aab485d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:49:20.107088 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-dcda4be8-ce05-49b2-823f-c6af8aab485d None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:49:20.108291 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-dcda4be8-ce05-49b2-823f-c6af8aab485d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:49:20.138639 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1117/2206] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:49:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 10:49:20.147600 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b02ab17-720f-4330-b016-81663d617834 None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:49:20.149633 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-2b02ab17-720f-4330-b016-81663d617834 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:49:20.199383 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1090/2207] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:49:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:49:20.204701 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716586e9-055b-4d1e-8ce3-54581fbd7f84 None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:49:20.204993 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716586e9-055b-4d1e-8ce3-54581fbd7f84 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:49:20.205272 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716586e9-055b-4d1e-8ce3-54581fbd7f84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:49:20.233715 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-716586e9-055b-4d1e-8ce3-54581fbd7f84 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:49:20.235277 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-716586e9-055b-4d1e-8ce3-54581fbd7f84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:49:20.245669 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1091/2208] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:49:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 10:49:20.246280 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b02ab17-720f-4330-b016-81663d617834 None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:49:20.250085 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716586e9-055b-4d1e-8ce3-54581fbd7f84 None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:49:20.263399 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c597c9d-5f67-4fe6-b6dc-4cc1bed377c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7922d0a1a7c4e808d0b2a80dca0da40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:49:20.264109 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c597c9d-5f67-4fe6-b6dc-4cc1bed377c9 service neutron] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:49:20.264457 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c597c9d-5f67-4fe6-b6dc-4cc1bed377c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:49:20.264746 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c597c9d-5f67-4fe6-b6dc-4cc1bed377c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:49:20.279499 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716586e9-055b-4d1e-8ce3-54581fbd7f84 None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:49:20.288541 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d49daa-21a6-476e-82e6-620fb297a78f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:49:20.289575 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d49daa-21a6-476e-82e6-620fb297a78f admin admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:49:20.289941 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d49daa-21a6-476e-82e6-620fb297a78f admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:49:20.290297 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d49daa-21a6-476e-82e6-620fb297a78f admin admin] PATH_INFO: `/v3/domains` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:49:20.290559 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c597c9d-5f67-4fe6-b6dc-4cc1bed377c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:49:20.291332 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d49daa-21a6-476e-82e6-620fb297a78f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86828) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:49:20.292369 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d49daa-21a6-476e-82e6-620fb297a78f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:49:20.294426 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d49daa-21a6-476e-82e6-620fb297a78f admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:49:20.294648 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c597c9d-5f67-4fe6-b6dc-4cc1bed377c9 service neutron] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:49:20.305667 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1118/2209] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:49:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 10:49:20.345738 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1092/2210] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Jul 27 10:49:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 10:49:20.347216 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c597c9d-5f67-4fe6-b6dc-4cc1bed377c9 service neutron] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:49:20.357523 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c597c9d-5f67-4fe6-b6dc-4cc1bed377c9 service neutron] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:49:20.363335 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50170edd-37bf-47f6-b7ce-5338463624b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:49:20.364267 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50170edd-37bf-47f6-b7ce-5338463624b2 admin admin] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:49:20.364735 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50170edd-37bf-47f6-b7ce-5338463624b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:49:20.365194 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50170edd-37bf-47f6-b7ce-5338463624b2 admin admin] PATH_INFO: `/v3/projects` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:49:20.366072 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50170edd-37bf-47f6-b7ce-5338463624b2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:49:20.368244 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50170edd-37bf-47f6-b7ce-5338463624b2 admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:49:20.420161 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1119/2211] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Jul 27 10:49:20 2022] POST /identity/v3/projects => generated 369 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 10:49:20.424241 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d49daa-21a6-476e-82e6-620fb297a78f admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:49:20.437666 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d49daa-21a6-476e-82e6-620fb297a78f admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:49:20.553468 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb4ca9d-7609-4ca6-8efa-93c3bdca43af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:49:20.554255 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb4ca9d-7609-4ca6-8efa-93c3bdca43af admin admin] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:49:20.554568 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb4ca9d-7609-4ca6-8efa-93c3bdca43af admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:49:20.554900 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb4ca9d-7609-4ca6-8efa-93c3bdca43af admin admin] PATH_INFO: `/v3/users` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:49:20.555570 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb4ca9d-7609-4ca6-8efa-93c3bdca43af admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:49:20.558905 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb4ca9d-7609-4ca6-8efa-93c3bdca43af admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:49:20.596389 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1093/2212] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Jul 27 10:49:20 2022] POST /identity/v3/users => generated 285 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 10:49:20.597378 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50170edd-37bf-47f6-b7ce-5338463624b2 admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:49:20.619333 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50170edd-37bf-47f6-b7ce-5338463624b2 admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:49:20.628463 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a414c8f6-ebef-41b5-9fe2-1e4c8d8df439 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:49:20.628880 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a414c8f6-ebef-41b5-9fe2-1e4c8d8df439 admin admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:49:20.629181 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a414c8f6-ebef-41b5-9fe2-1e4c8d8df439 admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:49:20.629464 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a414c8f6-ebef-41b5-9fe2-1e4c8d8df439 admin admin] PATH_INFO: `/v3/roles` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:49:20.630185 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a414c8f6-ebef-41b5-9fe2-1e4c8d8df439 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:49:20.631840 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a414c8f6-ebef-41b5-9fe2-1e4c8d8df439 admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:49:20.647117 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1120/2213] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 10:49:20 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 10:49:20.648788 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb4ca9d-7609-4ca6-8efa-93c3bdca43af admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:49:20.661936 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb4ca9d-7609-4ca6-8efa-93c3bdca43af admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:49:20.669462 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e663fa8b-4d94-41ff-9c5e-2a16c6e8f973 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:49:20.670193 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e663fa8b-4d94-41ff-9c5e-2a16c6e8f973 admin admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:49:20.671119 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e663fa8b-4d94-41ff-9c5e-2a16c6e8f973 admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:49:20.671119 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e663fa8b-4d94-41ff-9c5e-2a16c6e8f973 admin admin] PATH_INFO: `/v3/roles` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:49:20.672370 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e663fa8b-4d94-41ff-9c5e-2a16c6e8f973 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:49:20.674514 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e663fa8b-4d94-41ff-9c5e-2a16c6e8f973 admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:49:20.691110 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1094/2214] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 10:49:20 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 10:49:20.695006 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a414c8f6-ebef-41b5-9fe2-1e4c8d8df439 admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:49:20.715449 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a414c8f6-ebef-41b5-9fe2-1e4c8d8df439 admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:49:20.723571 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc14eb6-a845-47fa-8546-7cc0182f773a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:49:20.724428 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc14eb6-a845-47fa-8546-7cc0182f773a admin admin] REQUEST_METHOD: `PUT` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:49:20.725056 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc14eb6-a845-47fa-8546-7cc0182f773a admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:49:20.725347 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc14eb6-a845-47fa-8546-7cc0182f773a admin admin] PATH_INFO: `/v3/projects/a18de5ec57ea4d4794f2be3cca33c097/users/43050fb9e0904edaa7fe3e48b12000ed/roles/4c9fc0b3baed441da2126be41cc6b04a` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:49:20.747522 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc14eb6-a845-47fa-8546-7cc0182f773a admin admin] RBAC: Authorizing `identity:create_grant(project_id=a18de5ec57ea4d4794f2be3cca33c097, user_id=43050fb9e0904edaa7fe3e48b12000ed, role_id=4c9fc0b3baed441da2126be41cc6b04a)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:49:20.751454 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc14eb6-a845-47fa-8546-7cc0182f773a admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:49:20.786875 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1121/2215] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 10:49:20 2022] PUT /identity/v3/projects/a18de5ec57ea4d4794f2be3cca33c097/users/43050fb9e0904edaa7fe3e48b12000ed/roles/4c9fc0b3baed441da2126be41cc6b04a => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:49:20.794888 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4412e1a-8940-495c-b8f8-850755715811 None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:49:20.794888 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4412e1a-8940-495c-b8f8-850755715811 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:49:20.794888 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4412e1a-8940-495c-b8f8-850755715811 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:49:20.798579 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc14eb6-a845-47fa-8546-7cc0182f773a admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:49:20.805518 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-ebc14eb6-a845-47fa-8546-7cc0182f773a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:49:20.824682 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:DELETE=4,keystone:SELECT=93,keystone:INSERT=3 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:49:20.834470 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-f4412e1a-8940-495c-b8f8-850755715811 None None] MFA Rules not processed for user `43050fb9e0904edaa7fe3e48b12000ed`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:49:20.836609 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-f4412e1a-8940-495c-b8f8-850755715811 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:49:20.858810 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:49:20.950648 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1095/2216] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:49:20 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 10:49:20.951997 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc14eb6-a845-47fa-8546-7cc0182f773a admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:49:20.959330 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15af07c-8df7-4a38-af98-84ceaf7e6b0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:49:20.960005 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15af07c-8df7-4a38-af98-84ceaf7e6b0d admin admin] REQUEST_METHOD: `DELETE` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:49:20.960252 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15af07c-8df7-4a38-af98-84ceaf7e6b0d admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:49:20.960529 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15af07c-8df7-4a38-af98-84ceaf7e6b0d admin admin] PATH_INFO: `/v3/projects/3c288c531b5f431d819c0eafff040145` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:49:20.961572 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4412e1a-8940-495c-b8f8-850755715811 None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:49:20.961788 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15af07c-8df7-4a38-af98-84ceaf7e6b0d admin admin] RBAC: Authorizing `identity:delete_project(project_id=3c288c531b5f431d819c0eafff040145)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:49:20.964389 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15af07c-8df7-4a38-af98-84ceaf7e6b0d admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:49:20.964989 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-f4412e1a-8940-495c-b8f8-850755715811 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:49:21.060085 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4412e1a-8940-495c-b8f8-850755715811 None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:49:21.064906 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f0edf1-54a6-4825-8a72-6f93ed880c7c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc03c5d232654878aa1ddce15f49fdaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:49:21.065761 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f0edf1-54a6-4825-8a72-6f93ed880c7c service glance] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:49:21.066112 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f0edf1-54a6-4825-8a72-6f93ed880c7c service glance] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:49:21.066462 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f0edf1-54a6-4825-8a72-6f93ed880c7c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:49:21.079829 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f0edf1-54a6-4825-8a72-6f93ed880c7c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:49:21.082371 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f0edf1-54a6-4825-8a72-6f93ed880c7c service glance] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:49:21.101668 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-a15af07c-8df7-4a38-af98-84ceaf7e6b0d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3c288c531b5f431d819c0eafff040145', 'request_id': 'req-a15af07c-8df7-4a38-af98-84ceaf7e6b0d'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:49:21.119252 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1096/2217] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:49:20 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 10:49:21.120325 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-a15af07c-8df7-4a38-af98-84ceaf7e6b0d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3c288c531b5f431d819c0eafff040145', 'request_id': 'req-a15af07c-8df7-4a38-af98-84ceaf7e6b0d'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:49:21.131755 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-a15af07c-8df7-4a38-af98-84ceaf7e6b0d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3c288c531b5f431d819c0eafff040145', 'request_id': 'req-a15af07c-8df7-4a38-af98-84ceaf7e6b0d'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:49:21.161254 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-a15af07c-8df7-4a38-af98-84ceaf7e6b0d admin admin] The token cache is being invalidate because project 3c288c531b5f431d819c0eafff040145 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86827) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 10:49:21.161467 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-a15af07c-8df7-4a38-af98-84ceaf7e6b0d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:49:21.162155 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-a15af07c-8df7-4a38-af98-84ceaf7e6b0d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:49:21.163750 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1122/2218] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Jul 27 10:49:20 2022] DELETE /identity/v3/projects/3c288c531b5f431d819c0eafff040145 => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:49:21.182765 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3e55ec-ffa3-4384-9af8-14344b233f2c None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:49:21.183312 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3e55ec-ffa3-4384-9af8-14344b233f2c None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:49:21.183705 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3e55ec-ffa3-4384-9af8-14344b233f2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:49:21.219166 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-aa3e55ec-ffa3-4384-9af8-14344b233f2c None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:49:21.224391 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-aa3e55ec-ffa3-4384-9af8-14344b233f2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:49:21.330159 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1097/2219] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:49:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:49:21.337829 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8c2da1-9bf9-4e88-a66c-5aaf426ead5f None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:49:21.338461 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8c2da1-9bf9-4e88-a66c-5aaf426ead5f None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:49:21.338461 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8c2da1-9bf9-4e88-a66c-5aaf426ead5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:49:21.370306 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-8c8c2da1-9bf9-4e88-a66c-5aaf426ead5f None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:49:21.372151 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-8c8c2da1-9bf9-4e88-a66c-5aaf426ead5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:49:21.388456 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1123/2220] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:49:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:49:21.392728 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3e55ec-ffa3-4384-9af8-14344b233f2c None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:49:21.411977 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3e55ec-ffa3-4384-9af8-14344b233f2c None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:49:21.419752 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e3cf34-bd6c-4802-afb8-5ccdbfb404b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:49:21.420393 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e3cf34-bd6c-4802-afb8-5ccdbfb404b3 admin admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:49:21.420716 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e3cf34-bd6c-4802-afb8-5ccdbfb404b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:49:21.420993 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e3cf34-bd6c-4802-afb8-5ccdbfb404b3 admin admin] PATH_INFO: `/v3/domains` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:49:21.421856 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e3cf34-bd6c-4802-afb8-5ccdbfb404b3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86828) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:49:21.422174 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e3cf34-bd6c-4802-afb8-5ccdbfb404b3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:49:21.424158 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e3cf34-bd6c-4802-afb8-5ccdbfb404b3 admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:49:21.454652 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1098/2221] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Jul 27 10:49:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 10:49:21.457547 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8c2da1-9bf9-4e88-a66c-5aaf426ead5f None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:49:21.477643 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8c2da1-9bf9-4e88-a66c-5aaf426ead5f None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:49:21.487685 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c40e176-1fd9-4cc2-bc76-f913a14f74be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:49:21.488473 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c40e176-1fd9-4cc2-bc76-f913a14f74be admin admin] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:49:21.488766 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c40e176-1fd9-4cc2-bc76-f913a14f74be admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:49:21.489219 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c40e176-1fd9-4cc2-bc76-f913a14f74be admin admin] PATH_INFO: `/v3/projects` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:49:21.489928 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c40e176-1fd9-4cc2-bc76-f913a14f74be admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:49:21.500687 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c40e176-1fd9-4cc2-bc76-f913a14f74be admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:49:21.559332 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1124/2222] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Jul 27 10:49:21 2022] POST /identity/v3/projects => generated 373 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 10:49:21.560205 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e3cf34-bd6c-4802-afb8-5ccdbfb404b3 admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:49:21.575397 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e3cf34-bd6c-4802-afb8-5ccdbfb404b3 admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:49:21.649390 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:49:21.675640 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff261bca-c10d-47b4-83a2-ee7605410be8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:49:21.676528 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff261bca-c10d-47b4-83a2-ee7605410be8 admin admin] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:49:21.676861 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff261bca-c10d-47b4-83a2-ee7605410be8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:49:21.677145 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff261bca-c10d-47b4-83a2-ee7605410be8 admin admin] PATH_INFO: `/v3/users` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:49:21.679933 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff261bca-c10d-47b4-83a2-ee7605410be8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:49:21.682733 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff261bca-c10d-47b4-83a2-ee7605410be8 admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:49:21.734743 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1099/2223] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Jul 27 10:49:21 2022] POST /identity/v3/users => generated 287 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 10:49:21.738637 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c40e176-1fd9-4cc2-bc76-f913a14f74be admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:49:21.754198 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c40e176-1fd9-4cc2-bc76-f913a14f74be admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:49:21.759629 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244a5448-a237-42ec-82da-d795634625bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:49:21.761476 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244a5448-a237-42ec-82da-d795634625bc admin admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:49:21.761880 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244a5448-a237-42ec-82da-d795634625bc admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:49:21.762180 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244a5448-a237-42ec-82da-d795634625bc admin admin] PATH_INFO: `/v3/roles` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:49:21.765881 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244a5448-a237-42ec-82da-d795634625bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:49:21.765881 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244a5448-a237-42ec-82da-d795634625bc admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:49:21.807665 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff261bca-c10d-47b4-83a2-ee7605410be8 admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:49:21.811971 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1125/2224] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 10:49:21 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 10:49:21.830452 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff261bca-c10d-47b4-83a2-ee7605410be8 admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:49:21.839165 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffbf872-4a08-4f24-a5c7-90c8aa919dc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:49:21.841530 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffbf872-4a08-4f24-a5c7-90c8aa919dc3 admin admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:49:21.841530 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffbf872-4a08-4f24-a5c7-90c8aa919dc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:49:21.841530 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffbf872-4a08-4f24-a5c7-90c8aa919dc3 admin admin] PATH_INFO: `/v3/roles` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:49:21.842039 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffbf872-4a08-4f24-a5c7-90c8aa919dc3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:49:21.844529 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffbf872-4a08-4f24-a5c7-90c8aa919dc3 admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:49:21.858384 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1100/2225] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 10:49:21 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 10:49:21.863049 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244a5448-a237-42ec-82da-d795634625bc admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:49:21.876656 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244a5448-a237-42ec-82da-d795634625bc admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:49:21.882188 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3707fc5b-49e2-4e6a-b498-855919e0e40e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:49:21.882951 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3707fc5b-49e2-4e6a-b498-855919e0e40e admin admin] REQUEST_METHOD: `PUT` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:49:21.883236 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3707fc5b-49e2-4e6a-b498-855919e0e40e admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:49:21.883513 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3707fc5b-49e2-4e6a-b498-855919e0e40e admin admin] PATH_INFO: `/v3/projects/2a67572be8d9451b842921f9ead01a01/users/6b503e179737407590b3e84fd1255ab3/roles/4c9fc0b3baed441da2126be41cc6b04a` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:49:21.900229 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3707fc5b-49e2-4e6a-b498-855919e0e40e admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a67572be8d9451b842921f9ead01a01, user_id=6b503e179737407590b3e84fd1255ab3, role_id=4c9fc0b3baed441da2126be41cc6b04a)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:49:21.903490 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3707fc5b-49e2-4e6a-b498-855919e0e40e admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:49:21.920065 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1126/2226] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 10:49:21 2022] PUT /identity/v3/projects/2a67572be8d9451b842921f9ead01a01/users/6b503e179737407590b3e84fd1255ab3/roles/4c9fc0b3baed441da2126be41cc6b04a => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:49:21.931519 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5a8db2-476a-46ad-9ebf-8eb9ea6322a6 None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:49:21.931947 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5a8db2-476a-46ad-9ebf-8eb9ea6322a6 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:49:21.932218 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5a8db2-476a-46ad-9ebf-8eb9ea6322a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:49:21.945147 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3707fc5b-49e2-4e6a-b498-855919e0e40e admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:49:21.957044 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-3707fc5b-49e2-4e6a-b498-855919e0e40e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:49:21.989653 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-fd5a8db2-476a-46ad-9ebf-8eb9ea6322a6 None None] MFA Rules not processed for user `6b503e179737407590b3e84fd1255ab3`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:49:21.990845 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-fd5a8db2-476a-46ad-9ebf-8eb9ea6322a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:49:22.001822 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:49:22.045196 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3707fc5b-49e2-4e6a-b498-855919e0e40e admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:49:22.050021 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee89f80-ceb6-465e-972b-65c101d200e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:49:22.050702 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee89f80-ceb6-465e-972b-65c101d200e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:49:22.050990 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee89f80-ceb6-465e-972b-65c101d200e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:49:22.051270 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee89f80-ceb6-465e-972b-65c101d200e0 admin admin] PATH_INFO: `/v3/users/b82963d0f8f245aaa295379a38908c4d` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:49:22.052451 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee89f80-ceb6-465e-972b-65c101d200e0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b82963d0f8f245aaa295379a38908c4d)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:49:22.054388 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee89f80-ceb6-465e-972b-65c101d200e0 admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:49:22.066815 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1101/2227] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:49:21 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:49:22.081133 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5a8db2-476a-46ad-9ebf-8eb9ea6322a6 None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:49:22.082766 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-fd5a8db2-476a-46ad-9ebf-8eb9ea6322a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:49:22.120664 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-cee89f80-ceb6-465e-972b-65c101d200e0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b82963d0f8f245aaa295379a38908c4d'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:49:22.139750 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-cee89f80-ceb6-465e-972b-65c101d200e0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b82963d0f8f245aaa295379a38908c4d'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:49:22.154038 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-cee89f80-ceb6-465e-972b-65c101d200e0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b82963d0f8f245aaa295379a38908c4d'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:49:22.155652 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-cee89f80-ceb6-465e-972b-65c101d200e0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b82963d0f8f245aaa295379a38908c4d'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:49:22.157384 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-cee89f80-ceb6-465e-972b-65c101d200e0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b82963d0f8f245aaa295379a38908c4d'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:49:22.175667 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5a8db2-476a-46ad-9ebf-8eb9ea6322a6 None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:49:22.180756 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f991634-83ab-477f-89cd-1b6169167202 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '864d35f268cf45148756109759bb2743', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:49:22.181581 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f991634-83ab-477f-89cd-1b6169167202 service cinder] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:49:22.181866 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f991634-83ab-477f-89cd-1b6169167202 service cinder] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:49:22.182136 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f991634-83ab-477f-89cd-1b6169167202 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:49:22.184742 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-7f991634-83ab-477f-89cd-1b6169167202 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:49:22.185464 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1127/2228] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Jul 27 10:49:21 2022] DELETE /identity/v3/users/b82963d0f8f245aaa295379a38908c4d => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:49:22.202503 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f991634-83ab-477f-89cd-1b6169167202 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:49:22.205814 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f991634-83ab-477f-89cd-1b6169167202 service cinder] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:49:22.218156 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1102/2229] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:49:22 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 10:49:22.252028 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee89f80-ceb6-465e-972b-65c101d200e0 admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:49:22.253462 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-cee89f80-ceb6-465e-972b-65c101d200e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:49:22.334577 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee89f80-ceb6-465e-972b-65c101d200e0 admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:49:22.340171 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1960b37-b45b-47e2-a1b4-a2b8d1b8d1d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:49:22.340821 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1960b37-b45b-47e2-a1b4-a2b8d1b8d1d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:49:22.341133 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1960b37-b45b-47e2-a1b4-a2b8d1b8d1d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:49:22.341411 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1960b37-b45b-47e2-a1b4-a2b8d1b8d1d3 admin admin] PATH_INFO: `/v3/users/6b503e179737407590b3e84fd1255ab3` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:49:22.342745 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1960b37-b45b-47e2-a1b4-a2b8d1b8d1d3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6b503e179737407590b3e84fd1255ab3)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:49:22.344732 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1960b37-b45b-47e2-a1b4-a2b8d1b8d1d3 admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:49:22.401094 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-a1960b37-b45b-47e2-a1b4-a2b8d1b8d1d3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6b503e179737407590b3e84fd1255ab3'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:49:22.413999 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:49:22.416338 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-a1960b37-b45b-47e2-a1b4-a2b8d1b8d1d3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6b503e179737407590b3e84fd1255ab3'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:49:22.437584 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-a1960b37-b45b-47e2-a1b4-a2b8d1b8d1d3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6b503e179737407590b3e84fd1255ab3'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:49:22.438368 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-a1960b37-b45b-47e2-a1b4-a2b8d1b8d1d3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6b503e179737407590b3e84fd1255ab3'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:49:22.439118 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-a1960b37-b45b-47e2-a1b4-a2b8d1b8d1d3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6b503e179737407590b3e84fd1255ab3'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:49:22.465455 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1128/2230] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Jul 27 10:49:22 2022] DELETE /identity/v3/users/6b503e179737407590b3e84fd1255ab3 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:49:22.471243 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc7fb34-d661-4561-8acd-97dbefb79241 None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:49:22.471547 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc7fb34-d661-4561-8acd-97dbefb79241 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:49:22.471897 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc7fb34-d661-4561-8acd-97dbefb79241 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:49:22.502316 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-3fc7fb34-d661-4561-8acd-97dbefb79241 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:49:22.503798 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-3fc7fb34-d661-4561-8acd-97dbefb79241 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:49:22.570495 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1960b37-b45b-47e2-a1b4-a2b8d1b8d1d3 admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:49:22.572347 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-a1960b37-b45b-47e2-a1b4-a2b8d1b8d1d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:49:22.576947 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:49:22.593465 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1103/2231] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:49:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:49:22.605356 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc7fb34-d661-4561-8acd-97dbefb79241 None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:49:22.607025 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-3fc7fb34-d661-4561-8acd-97dbefb79241 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:49:22.684663 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc7fb34-d661-4561-8acd-97dbefb79241 None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:49:22.685468 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1960b37-b45b-47e2-a1b4-a2b8d1b8d1d3 admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:49:22.688621 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2ec220-b6da-442a-a8e3-a87a46f8e0ce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7922d0a1a7c4e808d0b2a80dca0da40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:49:22.689245 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2ec220-b6da-442a-a8e3-a87a46f8e0ce service neutron] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:49:22.689527 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2ec220-b6da-442a-a8e3-a87a46f8e0ce service neutron] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:49:22.689805 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2ec220-b6da-442a-a8e3-a87a46f8e0ce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:49:22.690631 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b85589-0d45-4a10-8d3f-6b5b21455d2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:49:22.691655 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b85589-0d45-4a10-8d3f-6b5b21455d2a admin admin] REQUEST_METHOD: `DELETE` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:49:22.691975 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b85589-0d45-4a10-8d3f-6b5b21455d2a admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:49:22.692365 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b85589-0d45-4a10-8d3f-6b5b21455d2a admin admin] PATH_INFO: `/v3/projects/728ed167367f4d99ad27e0b98c969e1a` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:49:22.693543 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b85589-0d45-4a10-8d3f-6b5b21455d2a admin admin] RBAC: Authorizing `identity:delete_project(project_id=728ed167367f4d99ad27e0b98c969e1a)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:49:22.695773 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b85589-0d45-4a10-8d3f-6b5b21455d2a admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:49:22.704419 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2ec220-b6da-442a-a8e3-a87a46f8e0ce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:49:22.705859 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2ec220-b6da-442a-a8e3-a87a46f8e0ce service neutron] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:49:22.716453 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1104/2232] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:49:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 10:49:22.752599 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-94b85589-0d45-4a10-8d3f-6b5b21455d2a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '728ed167367f4d99ad27e0b98c969e1a', 'request_id': 'req-94b85589-0d45-4a10-8d3f-6b5b21455d2a'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:49:22.759714 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-94b85589-0d45-4a10-8d3f-6b5b21455d2a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '728ed167367f4d99ad27e0b98c969e1a', 'request_id': 'req-94b85589-0d45-4a10-8d3f-6b5b21455d2a'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:49:22.772060 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-94b85589-0d45-4a10-8d3f-6b5b21455d2a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '728ed167367f4d99ad27e0b98c969e1a', 'request_id': 'req-94b85589-0d45-4a10-8d3f-6b5b21455d2a'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:49:22.783295 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-94b85589-0d45-4a10-8d3f-6b5b21455d2a admin admin] The token cache is being invalidate because project 728ed167367f4d99ad27e0b98c969e1a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86827) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 10:49:22.783552 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-94b85589-0d45-4a10-8d3f-6b5b21455d2a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:49:22.784192 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-94b85589-0d45-4a10-8d3f-6b5b21455d2a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:49:22.785855 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1129/2233] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Jul 27 10:49:22 2022] DELETE /identity/v3/projects/728ed167367f4d99ad27e0b98c969e1a => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:49:22.792377 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce78b01-67fd-4cc4-85b7-2ac7310dfc2d None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:49:22.792797 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce78b01-67fd-4cc4-85b7-2ac7310dfc2d None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:49:22.793081 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce78b01-67fd-4cc4-85b7-2ac7310dfc2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:49:22.822394 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-7ce78b01-67fd-4cc4-85b7-2ac7310dfc2d None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:49:22.823842 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-7ce78b01-67fd-4cc4-85b7-2ac7310dfc2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:49:22.908205 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1105/2234] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:49:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:49:22.916130 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891eb122-77b1-430e-b5bd-f918b8da77fe None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:49:22.916427 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891eb122-77b1-430e-b5bd-f918b8da77fe None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:49:22.917274 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891eb122-77b1-430e-b5bd-f918b8da77fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:49:22.946913 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-891eb122-77b1-430e-b5bd-f918b8da77fe None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:49:22.948570 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-891eb122-77b1-430e-b5bd-f918b8da77fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:49:22.966462 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1130/2235] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:49:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:49:22.971253 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce78b01-67fd-4cc4-85b7-2ac7310dfc2d None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:49:22.981817 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce78b01-67fd-4cc4-85b7-2ac7310dfc2d None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:49:22.987194 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0fef9b-da6d-4232-96aa-5722f7638fc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:49:22.987887 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0fef9b-da6d-4232-96aa-5722f7638fc7 admin admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:49:22.988255 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0fef9b-da6d-4232-96aa-5722f7638fc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:49:22.988609 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0fef9b-da6d-4232-96aa-5722f7638fc7 admin admin] PATH_INFO: `/v3/domains` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:49:22.989911 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0fef9b-da6d-4232-96aa-5722f7638fc7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86828) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:49:22.989911 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0fef9b-da6d-4232-96aa-5722f7638fc7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:49:22.994183 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0fef9b-da6d-4232-96aa-5722f7638fc7 admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:49:23.005381 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1106/2236] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Jul 27 10:49:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:49:23.010388 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891eb122-77b1-430e-b5bd-f918b8da77fe None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:49:23.020352 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891eb122-77b1-430e-b5bd-f918b8da77fe None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:49:23.025095 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f5bbc1-212f-4a4a-94bc-2724daedc45f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:49:23.025720 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f5bbc1-212f-4a4a-94bc-2724daedc45f admin admin] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:49:23.026008 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f5bbc1-212f-4a4a-94bc-2724daedc45f admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:49:23.026316 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f5bbc1-212f-4a4a-94bc-2724daedc45f admin admin] PATH_INFO: `/v3/projects` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:49:23.027026 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f5bbc1-212f-4a4a-94bc-2724daedc45f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:49:23.032429 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f5bbc1-212f-4a4a-94bc-2724daedc45f admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:49:23.064816 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0fef9b-da6d-4232-96aa-5722f7638fc7 admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:49:23.067535 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-dc0fef9b-da6d-4232-96aa-5722f7638fc7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:49:23.084241 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1131/2237] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Jul 27 10:49:23 2022] POST /identity/v3/projects => generated 389 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 10:49:23.086200 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f5bbc1-212f-4a4a-94bc-2724daedc45f admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:49:23.098020 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0fef9b-da6d-4232-96aa-5722f7638fc7 admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:49:23.140746 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:49:23.197639 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f5bbc1-212f-4a4a-94bc-2724daedc45f admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:49:23.198379 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd689dc9-0e93-4f71-a493-3d1b1b441ed1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:49:23.199384 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd689dc9-0e93-4f71-a493-3d1b1b441ed1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:49:23.199766 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd689dc9-0e93-4f71-a493-3d1b1b441ed1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:49:23.200215 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd689dc9-0e93-4f71-a493-3d1b1b441ed1 admin admin] PATH_INFO: `/v3/projects/2a67572be8d9451b842921f9ead01a01` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:49:23.203596 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b4b548-7ff1-4320-b879-7285d368c6c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:49:23.204217 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b4b548-7ff1-4320-b879-7285d368c6c4 admin admin] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:49:23.204498 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b4b548-7ff1-4320-b879-7285d368c6c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:49:23.204775 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b4b548-7ff1-4320-b879-7285d368c6c4 admin admin] PATH_INFO: `/v3/users` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:49:23.205460 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b4b548-7ff1-4320-b879-7285d368c6c4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:49:23.207445 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b4b548-7ff1-4320-b879-7285d368c6c4 admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:49:23.208163 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd689dc9-0e93-4f71-a493-3d1b1b441ed1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2a67572be8d9451b842921f9ead01a01)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:49:23.210420 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd689dc9-0e93-4f71-a493-3d1b1b441ed1 admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:49:23.246779 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1132/2238] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Jul 27 10:49:23 2022] POST /identity/v3/users => generated 295 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 10:49:23.250493 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b4b548-7ff1-4320-b879-7285d368c6c4 admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:49:23.268807 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b4b548-7ff1-4320-b879-7285d368c6c4 admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:49:23.270464 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:49:23.293273 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-cd689dc9-0e93-4f71-a493-3d1b1b441ed1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2a67572be8d9451b842921f9ead01a01', 'request_id': 'req-cd689dc9-0e93-4f71-a493-3d1b1b441ed1'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:49:23.302405 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-cd689dc9-0e93-4f71-a493-3d1b1b441ed1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2a67572be8d9451b842921f9ead01a01', 'request_id': 'req-cd689dc9-0e93-4f71-a493-3d1b1b441ed1'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:49:23.314545 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-cd689dc9-0e93-4f71-a493-3d1b1b441ed1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2a67572be8d9451b842921f9ead01a01', 'request_id': 'req-cd689dc9-0e93-4f71-a493-3d1b1b441ed1'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:49:23.333724 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-cd689dc9-0e93-4f71-a493-3d1b1b441ed1 admin admin] The token cache is being invalidate because project 2a67572be8d9451b842921f9ead01a01 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86828) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 10:49:23.334164 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-cd689dc9-0e93-4f71-a493-3d1b1b441ed1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:49:23.335100 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-cd689dc9-0e93-4f71-a493-3d1b1b441ed1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:49:23.341073 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1107/2239] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Jul 27 10:49:23 2022] DELETE /identity/v3/projects/2a67572be8d9451b842921f9ead01a01 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 10:49:23.343181 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e19f20-6ad4-4632-bce6-75498fede210 None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:49:23.343574 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e19f20-6ad4-4632-bce6-75498fede210 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:49:23.343965 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e19f20-6ad4-4632-bce6-75498fede210 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:49:23.351024 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7873d803-6db4-47ff-9428-ea34cbf14b89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:49:23.351833 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7873d803-6db4-47ff-9428-ea34cbf14b89 admin admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:49:23.352175 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7873d803-6db4-47ff-9428-ea34cbf14b89 admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:49:23.352451 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7873d803-6db4-47ff-9428-ea34cbf14b89 admin admin] PATH_INFO: `/v3/roles` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:49:23.353323 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7873d803-6db4-47ff-9428-ea34cbf14b89 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:49:23.356148 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7873d803-6db4-47ff-9428-ea34cbf14b89 admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:49:23.370733 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1133/2240] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 10:49:23 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 10:49:23.373387 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-25e19f20-6ad4-4632-bce6-75498fede210 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:49:23.374808 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-25e19f20-6ad4-4632-bce6-75498fede210 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:49:23.375966 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7873d803-6db4-47ff-9428-ea34cbf14b89 admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:49:23.378046 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-7873d803-6db4-47ff-9428-ea34cbf14b89 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:49:23.394636 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1108/2241] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:49:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:49:23.395682 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7873d803-6db4-47ff-9428-ea34cbf14b89 admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:49:23.399602 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb00be8d-bdf4-4a5f-978a-3a2a0feb93e2 None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:49:23.400170 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb00be8d-bdf4-4a5f-978a-3a2a0feb93e2 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:49:23.400595 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4454db08-450e-4f26-9ab5-a8d2f77e4937 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:49:23.400757 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb00be8d-bdf4-4a5f-978a-3a2a0feb93e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:49:23.401771 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4454db08-450e-4f26-9ab5-a8d2f77e4937 admin admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:49:23.402062 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4454db08-450e-4f26-9ab5-a8d2f77e4937 admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:49:23.403006 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4454db08-450e-4f26-9ab5-a8d2f77e4937 admin admin] PATH_INFO: `/v3/roles` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:49:23.403784 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4454db08-450e-4f26-9ab5-a8d2f77e4937 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:49:23.405418 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4454db08-450e-4f26-9ab5-a8d2f77e4937 admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:49:23.421722 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1134/2242] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 10:49:23 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 10:49:23.423140 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4454db08-450e-4f26-9ab5-a8d2f77e4937 admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:49:23.431625 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-bb00be8d-bdf4-4a5f-978a-3a2a0feb93e2 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:49:23.434430 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4454db08-450e-4f26-9ab5-a8d2f77e4937 admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:49:23.434626 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-bb00be8d-bdf4-4a5f-978a-3a2a0feb93e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:49:23.439385 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f7c6c7-12a4-4573-bbe7-d955a14893fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:49:23.440121 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f7c6c7-12a4-4573-bbe7-d955a14893fa admin admin] REQUEST_METHOD: `PUT` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:49:23.440359 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f7c6c7-12a4-4573-bbe7-d955a14893fa admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:49:23.440636 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f7c6c7-12a4-4573-bbe7-d955a14893fa admin admin] PATH_INFO: `/v3/projects/1f86323e8542471190e0b58769416138/users/fc1270b519b44320a772d7ad163c873d/roles/4c9fc0b3baed441da2126be41cc6b04a` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:49:23.444470 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1109/2243] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:49:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 10:49:23.450847 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb00be8d-bdf4-4a5f-978a-3a2a0feb93e2 None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:49:23.460235 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f7c6c7-12a4-4573-bbe7-d955a14893fa admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f86323e8542471190e0b58769416138, user_id=fc1270b519b44320a772d7ad163c873d, role_id=4c9fc0b3baed441da2126be41cc6b04a)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:49:23.462965 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f7c6c7-12a4-4573-bbe7-d955a14893fa admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:49:23.464803 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb00be8d-bdf4-4a5f-978a-3a2a0feb93e2 None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:49:23.474364 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259690d8-a787-440c-84d2-24a7b2fa88f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:49:23.475151 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259690d8-a787-440c-84d2-24a7b2fa88f0 admin admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:49:23.476118 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259690d8-a787-440c-84d2-24a7b2fa88f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:49:23.476406 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259690d8-a787-440c-84d2-24a7b2fa88f0 admin admin] PATH_INFO: `/v3/domains` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:49:23.477129 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259690d8-a787-440c-84d2-24a7b2fa88f0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86828) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:49:23.477446 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259690d8-a787-440c-84d2-24a7b2fa88f0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:49:23.479522 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259690d8-a787-440c-84d2-24a7b2fa88f0 admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:49:23.486664 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1135/2244] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 10:49:23 2022] PUT /identity/v3/projects/1f86323e8542471190e0b58769416138/users/fc1270b519b44320a772d7ad163c873d/roles/4c9fc0b3baed441da2126be41cc6b04a => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:49:23.488521 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072a7331-e44a-468d-a004-b52b109d2f7d None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:49:23.488521 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072a7331-e44a-468d-a004-b52b109d2f7d None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:49:23.488521 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072a7331-e44a-468d-a004-b52b109d2f7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:49:23.496707 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1110/2245] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Jul 27 10:49:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 10:49:23.499904 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259690d8-a787-440c-84d2-24a7b2fa88f0 admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:49:23.509147 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-072a7331-e44a-468d-a004-b52b109d2f7d None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:49:23.510212 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259690d8-a787-440c-84d2-24a7b2fa88f0 admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:49:23.511397 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-072a7331-e44a-468d-a004-b52b109d2f7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:49:23.594410 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a582e7eb-7967-45a6-8b5e-ff51dfd95ec1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:49:23.595100 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a582e7eb-7967-45a6-8b5e-ff51dfd95ec1 admin admin] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:49:23.595381 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a582e7eb-7967-45a6-8b5e-ff51dfd95ec1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:49:23.595679 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a582e7eb-7967-45a6-8b5e-ff51dfd95ec1 admin admin] PATH_INFO: `/v3/projects` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:49:23.596343 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a582e7eb-7967-45a6-8b5e-ff51dfd95ec1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:49:23.598720 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1136/2246] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:49:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 10:49:23.598889 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a582e7eb-7967-45a6-8b5e-ff51dfd95ec1 admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:49:23.606161 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072a7331-e44a-468d-a004-b52b109d2f7d None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:49:23.608563 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-072a7331-e44a-468d-a004-b52b109d2f7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:49:23.631297 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1111/2247] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Jul 27 10:49:23 2022] POST /identity/v3/projects => generated 389 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 10:49:23.635011 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a582e7eb-7967-45a6-8b5e-ff51dfd95ec1 admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:49:23.647613 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:49:23.695583 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072a7331-e44a-468d-a004-b52b109d2f7d None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:49:23.763336 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a582e7eb-7967-45a6-8b5e-ff51dfd95ec1 admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:49:23.768676 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06894fbe-6ae6-4dff-bf7a-bb6f3567bdd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:49:23.769320 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06894fbe-6ae6-4dff-bf7a-bb6f3567bdd1 admin admin] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:49:23.769603 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06894fbe-6ae6-4dff-bf7a-bb6f3567bdd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:49:23.769876 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06894fbe-6ae6-4dff-bf7a-bb6f3567bdd1 admin admin] PATH_INFO: `/v3/users` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:49:23.770547 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06894fbe-6ae6-4dff-bf7a-bb6f3567bdd1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:49:23.772628 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06894fbe-6ae6-4dff-bf7a-bb6f3567bdd1 admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:49:23.781311 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41702cf3-c68e-42b5-a51b-10a6824d9ad7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7922d0a1a7c4e808d0b2a80dca0da40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:49:23.782044 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41702cf3-c68e-42b5-a51b-10a6824d9ad7 service neutron] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:49:23.782813 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41702cf3-c68e-42b5-a51b-10a6824d9ad7 service neutron] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:49:23.783095 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41702cf3-c68e-42b5-a51b-10a6824d9ad7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:49:23.796336 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41702cf3-c68e-42b5-a51b-10a6824d9ad7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:49:23.797939 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1112/2248] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Jul 27 10:49:23 2022] POST /identity/v3/users => generated 295 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 10:49:23.798071 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41702cf3-c68e-42b5-a51b-10a6824d9ad7 service neutron] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:49:23.801712 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06894fbe-6ae6-4dff-bf7a-bb6f3567bdd1 admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:49:23.811486 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1137/2249] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:49:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 10:49:23.818775 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06894fbe-6ae6-4dff-bf7a-bb6f3567bdd1 admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:49:23.824898 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fd373a-d458-47a4-90b9-d18386ea75b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:49:23.825673 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fd373a-d458-47a4-90b9-d18386ea75b5 admin admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:49:23.825960 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fd373a-d458-47a4-90b9-d18386ea75b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:49:23.827203 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fd373a-d458-47a4-90b9-d18386ea75b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:49:23.827455 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fd373a-d458-47a4-90b9-d18386ea75b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:49:23.829114 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fd373a-d458-47a4-90b9-d18386ea75b5 admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:49:23.830817 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=90,keystone:DELETE=5,keystone:INSERT=5 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:49:23.843806 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1113/2250] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 10:49:23 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 10:49:23.847962 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41702cf3-c68e-42b5-a51b-10a6824d9ad7 service neutron] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:49:23.862688 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41702cf3-c68e-42b5-a51b-10a6824d9ad7 service neutron] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:49:23.868093 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecba738f-c77f-4897-ba54-792c6079ca37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:49:23.868334 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecba738f-c77f-4897-ba54-792c6079ca37 admin admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:49:23.868558 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecba738f-c77f-4897-ba54-792c6079ca37 admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:49:23.868890 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecba738f-c77f-4897-ba54-792c6079ca37 admin admin] PATH_INFO: `/v3/roles` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:49:23.870670 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecba738f-c77f-4897-ba54-792c6079ca37 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:49:23.871903 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecba738f-c77f-4897-ba54-792c6079ca37 admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:49:23.886379 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1138/2251] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 10:49:23 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 10:49:23.890815 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fd373a-d458-47a4-90b9-d18386ea75b5 admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:49:23.904518 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fd373a-d458-47a4-90b9-d18386ea75b5 admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:49:23.917677 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a687a245-210d-4269-a542-a67b8fe2df09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:49:23.919169 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a687a245-210d-4269-a542-a67b8fe2df09 admin admin] REQUEST_METHOD: `PUT` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:49:23.922537 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a687a245-210d-4269-a542-a67b8fe2df09 admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:49:23.922845 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a687a245-210d-4269-a542-a67b8fe2df09 admin admin] PATH_INFO: `/v3/projects/b00ee024e61e42eaa5d569cb7e2ff4d9/users/f00fabfc437e49259a46e3e4ae1760e5/roles/4c9fc0b3baed441da2126be41cc6b04a` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:49:23.948445 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a687a245-210d-4269-a542-a67b8fe2df09 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b00ee024e61e42eaa5d569cb7e2ff4d9, user_id=f00fabfc437e49259a46e3e4ae1760e5, role_id=4c9fc0b3baed441da2126be41cc6b04a)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:49:23.956299 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a687a245-210d-4269-a542-a67b8fe2df09 admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:49:23.996934 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307411d0-819a-47a1-bb50-0ac9bf2c024e None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:49:23.996934 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307411d0-819a-47a1-bb50-0ac9bf2c024e None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:49:23.997589 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307411d0-819a-47a1-bb50-0ac9bf2c024e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:49:23.999897 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1114/2252] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 10:49:23 2022] PUT /identity/v3/projects/b00ee024e61e42eaa5d569cb7e2ff4d9/users/f00fabfc437e49259a46e3e4ae1760e5/roles/4c9fc0b3baed441da2126be41cc6b04a => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:49:24.042749 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-307411d0-819a-47a1-bb50-0ac9bf2c024e None None] MFA Rules not processed for user `f00fabfc437e49259a46e3e4ae1760e5`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:49:24.045196 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-307411d0-819a-47a1-bb50-0ac9bf2c024e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:49:24.165662 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1139/2253] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:49:23 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:49:24.183300 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a687a245-210d-4269-a542-a67b8fe2df09 admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:49:24.185294 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-a687a245-210d-4269-a542-a67b8fe2df09 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:49:24.276257 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a687a245-210d-4269-a542-a67b8fe2df09 admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:49:24.280601 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf6ad15-7032-4760-8a65-e5fcfb1ce21a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '864d35f268cf45148756109759bb2743', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:49:24.281236 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf6ad15-7032-4760-8a65-e5fcfb1ce21a service cinder] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:49:24.281527 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf6ad15-7032-4760-8a65-e5fcfb1ce21a service cinder] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:49:24.281803 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf6ad15-7032-4760-8a65-e5fcfb1ce21a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:49:24.298853 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf6ad15-7032-4760-8a65-e5fcfb1ce21a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:49:24.300373 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf6ad15-7032-4760-8a65-e5fcfb1ce21a service cinder] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:49:24.310687 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1115/2254] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:49:24 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 10:49:28.805126 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3a6b8d-e112-4050-8b60-e1cb1e0d777a None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:49:28.805606 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3a6b8d-e112-4050-8b60-e1cb1e0d777a None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:49:28.805699 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3a6b8d-e112-4050-8b60-e1cb1e0d777a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:49:28.833775 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-0c3a6b8d-e112-4050-8b60-e1cb1e0d777a None None] MFA Rules not processed for user `fc1270b519b44320a772d7ad163c873d`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:49:28.834960 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-0c3a6b8d-e112-4050-8b60-e1cb1e0d777a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:49:28.884836 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:49:28.903652 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1140/2255] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:49:28 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 10:49:28.918466 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf6ad15-7032-4760-8a65-e5fcfb1ce21a service cinder] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:49:28.920592 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-9bf6ad15-7032-4760-8a65-e5fcfb1ce21a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:49:29.036454 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf6ad15-7032-4760-8a65-e5fcfb1ce21a service cinder] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:49:29.042966 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f2003d-910a-4757-b134-61626ee43018 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e059d8b6c28c4fcba925d05d7c98e4ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:49:29.043739 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f2003d-910a-4757-b134-61626ee43018 service nova] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:49:29.044127 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f2003d-910a-4757-b134-61626ee43018 service nova] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:49:29.044505 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f2003d-910a-4757-b134-61626ee43018 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:49:29.054853 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f2003d-910a-4757-b134-61626ee43018 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:49:29.056666 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f2003d-910a-4757-b134-61626ee43018 service nova] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:49:29.067411 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1116/2256] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:49:28 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 10:49:31.022198 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e82455c-34c1-4189-9ef5-0cceafc7e990 None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:49:31.022664 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e82455c-34c1-4189-9ef5-0cceafc7e990 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:49:31.023048 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e82455c-34c1-4189-9ef5-0cceafc7e990 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:49:31.044068 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-3e82455c-34c1-4189-9ef5-0cceafc7e990 None None] MFA Rules not processed for user `fc1270b519b44320a772d7ad163c873d`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:49:31.045307 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-3e82455c-34c1-4189-9ef5-0cceafc7e990 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:49:31.053924 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1141/2257] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:49:31 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:49:39.063369 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=72,keystone:INSERT=1 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:49:41.038226 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=7 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:49:41.485567 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb536106-8963-4e27-b36a-bbf5220ebf2b None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:49:41.486065 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb536106-8963-4e27-b36a-bbf5220ebf2b None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:49:41.486159 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb536106-8963-4e27-b36a-bbf5220ebf2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:49:41.515778 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-cb536106-8963-4e27-b36a-bbf5220ebf2b None None] MFA Rules not processed for user `3d9fed24b4ba44b0b5961a603c4362c4`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:49:41.517120 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-cb536106-8963-4e27-b36a-bbf5220ebf2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:49:41.572916 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1117/2258] 10.222.0.36 () {58 vars in 1009 bytes} [Wed Jul 27 10:49:41 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:49:41.622445 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e82455c-34c1-4189-9ef5-0cceafc7e990 None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:49:41.623888 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-3e82455c-34c1-4189-9ef5-0cceafc7e990 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:49:41.697676 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e82455c-34c1-4189-9ef5-0cceafc7e990 None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:49:41.702884 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6baf3f1-4730-47fc-987d-66bae42ef5c6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89ac506eefcd4e42a84b44fa15fb3975', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:49:41.703515 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6baf3f1-4730-47fc-987d-66bae42ef5c6 service swift] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:49:41.703793 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6baf3f1-4730-47fc-987d-66bae42ef5c6 service swift] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:49:41.704066 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6baf3f1-4730-47fc-987d-66bae42ef5c6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:49:41.712065 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6baf3f1-4730-47fc-987d-66bae42ef5c6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:49:41.713628 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6baf3f1-4730-47fc-987d-66bae42ef5c6 service swift] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:49:41.721490 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1142/2259] 10.222.0.36 () {60 vars in 1384 bytes} [Wed Jul 27 10:49:41 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 27 10:49:42.353802 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48127689-aa35-46ba-8e11-d219046c1131 None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:49:42.354160 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48127689-aa35-46ba-8e11-d219046c1131 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:49:42.354508 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48127689-aa35-46ba-8e11-d219046c1131 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:49:42.378696 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-48127689-aa35-46ba-8e11-d219046c1131 None None] MFA Rules not processed for user `3d9fed24b4ba44b0b5961a603c4362c4`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:49:42.380255 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-48127689-aa35-46ba-8e11-d219046c1131 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:49:42.405352 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1118/2260] 10.222.0.36 () {58 vars in 967 bytes} [Wed Jul 27 10:49:42 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 10:49:42.406128 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6baf3f1-4730-47fc-987d-66bae42ef5c6 service swift] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:49:42.419025 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6baf3f1-4730-47fc-987d-66bae42ef5c6 service swift] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:49:42.426793 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dac4573-e01d-4ff4-bef7-6121cb711558 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89ac506eefcd4e42a84b44fa15fb3975', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:49:42.426793 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dac4573-e01d-4ff4-bef7-6121cb711558 service swift] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:49:42.426793 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dac4573-e01d-4ff4-bef7-6121cb711558 service swift] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:49:42.426793 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dac4573-e01d-4ff4-bef7-6121cb711558 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:49:42.444956 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dac4573-e01d-4ff4-bef7-6121cb711558 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:49:42.446608 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dac4573-e01d-4ff4-bef7-6121cb711558 service swift] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:49:42.455938 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1143/2261] 10.222.0.36 () {60 vars in 1384 bytes} [Wed Jul 27 10:49:42 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 27 10:49:42.706184 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308d2b6a-886b-4800-bff4-0c2269fce670 None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:49:42.706679 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308d2b6a-886b-4800-bff4-0c2269fce670 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:49:42.706819 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308d2b6a-886b-4800-bff4-0c2269fce670 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:49:42.726756 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-308d2b6a-886b-4800-bff4-0c2269fce670 None None] MFA Rules not processed for user `3d9fed24b4ba44b0b5961a603c4362c4`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:49:42.727950 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-308d2b6a-886b-4800-bff4-0c2269fce670 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:49:42.736480 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1119/2262] 10.222.0.36 () {58 vars in 1009 bytes} [Wed Jul 27 10:49:42 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:49:42.748821 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dac4573-e01d-4ff4-bef7-6121cb711558 service swift] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:49:42.759756 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dac4573-e01d-4ff4-bef7-6121cb711558 service swift] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:49:42.765067 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989c0c88-85e8-4e08-8813-52583ae66671 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89ac506eefcd4e42a84b44fa15fb3975', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:49:42.765712 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989c0c88-85e8-4e08-8813-52583ae66671 service swift] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:49:42.765997 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989c0c88-85e8-4e08-8813-52583ae66671 service swift] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:49:42.766279 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989c0c88-85e8-4e08-8813-52583ae66671 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:49:42.774877 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989c0c88-85e8-4e08-8813-52583ae66671 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:49:42.776461 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989c0c88-85e8-4e08-8813-52583ae66671 service swift] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:49:42.785518 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1144/2263] 10.222.0.36 () {60 vars in 1384 bytes} [Wed Jul 27 10:49:42 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 27 10:49:43.146114 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd18454b-bbd7-4523-b682-d816aa8fe336 None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:49:43.146525 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd18454b-bbd7-4523-b682-d816aa8fe336 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:49:43.146999 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd18454b-bbd7-4523-b682-d816aa8fe336 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:49:43.166983 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-fd18454b-bbd7-4523-b682-d816aa8fe336 None None] MFA Rules not processed for user `3d9fed24b4ba44b0b5961a603c4362c4`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:49:43.168737 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-fd18454b-bbd7-4523-b682-d816aa8fe336 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:49:43.176305 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1120/2264] 10.222.0.36 () {58 vars in 967 bytes} [Wed Jul 27 10:49:43 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:49:43.189331 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989c0c88-85e8-4e08-8813-52583ae66671 service swift] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:49:43.200321 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989c0c88-85e8-4e08-8813-52583ae66671 service swift] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:49:43.205604 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627f8917-1441-4502-9927-f87cd5857358 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89ac506eefcd4e42a84b44fa15fb3975', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:49:43.206415 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627f8917-1441-4502-9927-f87cd5857358 service swift] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:49:43.206725 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627f8917-1441-4502-9927-f87cd5857358 service swift] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:49:43.207007 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627f8917-1441-4502-9927-f87cd5857358 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:49:43.215990 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627f8917-1441-4502-9927-f87cd5857358 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:49:43.217586 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627f8917-1441-4502-9927-f87cd5857358 service swift] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:49:43.225745 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1145/2265] 10.222.0.36 () {60 vars in 1384 bytes} [Wed Jul 27 10:49:43 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 27 10:49:50.063543 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd18454b-bbd7-4523-b682-d816aa8fe336 None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:49:50.139654 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd18454b-bbd7-4523-b682-d816aa8fe336 None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:49:50.147038 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4448303-c103-40dd-a13e-dde07556c0c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:49:50.148135 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4448303-c103-40dd-a13e-dde07556c0c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:49:50.148643 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4448303-c103-40dd-a13e-dde07556c0c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:49:50.149154 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4448303-c103-40dd-a13e-dde07556c0c3 admin admin] PATH_INFO: `/v3/users/f00fabfc437e49259a46e3e4ae1760e5` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:49:50.153399 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4448303-c103-40dd-a13e-dde07556c0c3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f00fabfc437e49259a46e3e4ae1760e5)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:49:50.157368 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4448303-c103-40dd-a13e-dde07556c0c3 admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:49:50.245276 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-d4448303-c103-40dd-a13e-dde07556c0c3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f00fabfc437e49259a46e3e4ae1760e5'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:49:50.260229 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-d4448303-c103-40dd-a13e-dde07556c0c3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f00fabfc437e49259a46e3e4ae1760e5'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:49:50.273845 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-d4448303-c103-40dd-a13e-dde07556c0c3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f00fabfc437e49259a46e3e4ae1760e5'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:49:50.274459 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-d4448303-c103-40dd-a13e-dde07556c0c3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f00fabfc437e49259a46e3e4ae1760e5'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:49:50.275718 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-d4448303-c103-40dd-a13e-dde07556c0c3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f00fabfc437e49259a46e3e4ae1760e5'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:49:50.295162 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1121/2266] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Jul 27 10:49:50 2022] DELETE /identity/v3/users/f00fabfc437e49259a46e3e4ae1760e5 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:49:50.299906 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3cf68e-6231-4c37-9ced-b44f0bf283a7 None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:49:50.300371 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3cf68e-6231-4c37-9ced-b44f0bf283a7 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:49:50.300662 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3cf68e-6231-4c37-9ced-b44f0bf283a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:49:50.320847 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-6d3cf68e-6231-4c37-9ced-b44f0bf283a7 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:49:50.322065 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-6d3cf68e-6231-4c37-9ced-b44f0bf283a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:49:50.384521 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1146/2267] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:49:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:49:50.394087 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4448303-c103-40dd-a13e-dde07556c0c3 admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:49:50.395532 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-d4448303-c103-40dd-a13e-dde07556c0c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:49:50.402925 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:49:50.461056 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4448303-c103-40dd-a13e-dde07556c0c3 admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:49:50.465473 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5471b935-a7bc-4180-88f3-baeba5972aa0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7922d0a1a7c4e808d0b2a80dca0da40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:49:50.466090 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5471b935-a7bc-4180-88f3-baeba5972aa0 service neutron] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:49:50.466372 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5471b935-a7bc-4180-88f3-baeba5972aa0 service neutron] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:49:50.466681 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5471b935-a7bc-4180-88f3-baeba5972aa0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:49:50.475810 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5471b935-a7bc-4180-88f3-baeba5972aa0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:49:50.477233 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5471b935-a7bc-4180-88f3-baeba5972aa0 service neutron] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:49:50.486752 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1122/2268] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:49:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 10:49:50.762748 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3cf68e-6231-4c37-9ced-b44f0bf283a7 None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:49:50.764336 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-6d3cf68e-6231-4c37-9ced-b44f0bf283a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:49:50.779677 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3cf68e-6231-4c37-9ced-b44f0bf283a7 None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:49:50.784326 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892fcccc-9fd9-42a1-a3fc-1251a010acde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:49:50.785864 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892fcccc-9fd9-42a1-a3fc-1251a010acde admin admin] REQUEST_METHOD: `DELETE` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:49:50.786509 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892fcccc-9fd9-42a1-a3fc-1251a010acde admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:49:50.786927 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892fcccc-9fd9-42a1-a3fc-1251a010acde admin admin] PATH_INFO: `/v3/projects/b00ee024e61e42eaa5d569cb7e2ff4d9` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:49:50.788161 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892fcccc-9fd9-42a1-a3fc-1251a010acde admin admin] RBAC: Authorizing `identity:delete_project(project_id=b00ee024e61e42eaa5d569cb7e2ff4d9)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:49:50.790400 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892fcccc-9fd9-42a1-a3fc-1251a010acde admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:49:50.839096 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:49:50.843633 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-892fcccc-9fd9-42a1-a3fc-1251a010acde admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b00ee024e61e42eaa5d569cb7e2ff4d9', 'request_id': 'req-892fcccc-9fd9-42a1-a3fc-1251a010acde'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:49:50.851243 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-892fcccc-9fd9-42a1-a3fc-1251a010acde admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b00ee024e61e42eaa5d569cb7e2ff4d9', 'request_id': 'req-892fcccc-9fd9-42a1-a3fc-1251a010acde'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:49:50.859525 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-892fcccc-9fd9-42a1-a3fc-1251a010acde admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b00ee024e61e42eaa5d569cb7e2ff4d9', 'request_id': 'req-892fcccc-9fd9-42a1-a3fc-1251a010acde'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:49:50.875388 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-892fcccc-9fd9-42a1-a3fc-1251a010acde admin admin] The token cache is being invalidate because project b00ee024e61e42eaa5d569cb7e2ff4d9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86827) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 10:49:50.875764 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-892fcccc-9fd9-42a1-a3fc-1251a010acde admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:49:50.876819 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-892fcccc-9fd9-42a1-a3fc-1251a010acde admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:49:50.878466 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1147/2269] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Jul 27 10:49:50 2022] DELETE /identity/v3/projects/b00ee024e61e42eaa5d569cb7e2ff4d9 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 10:49:50.883561 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1479390-ca70-4a56-a38f-138c88bab751 None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:49:50.884258 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1479390-ca70-4a56-a38f-138c88bab751 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:49:50.884258 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1479390-ca70-4a56-a38f-138c88bab751 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:49:50.905077 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-c1479390-ca70-4a56-a38f-138c88bab751 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:49:50.906291 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-c1479390-ca70-4a56-a38f-138c88bab751 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:49:50.976833 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1123/2270] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:49:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:49:50.984245 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80075d95-47ef-4f52-905a-45176d001453 None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:49:50.985003 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80075d95-47ef-4f52-905a-45176d001453 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:49:50.985003 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80075d95-47ef-4f52-905a-45176d001453 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:49:51.008339 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-80075d95-47ef-4f52-905a-45176d001453 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:49:51.009560 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-80075d95-47ef-4f52-905a-45176d001453 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:49:51.018441 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1148/2271] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:49:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:49:51.022825 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1479390-ca70-4a56-a38f-138c88bab751 None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:49:51.032851 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1479390-ca70-4a56-a38f-138c88bab751 None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:49:51.048453 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7d604e-37bc-4b2d-885f-59e1f0783703 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:49:51.049106 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7d604e-37bc-4b2d-885f-59e1f0783703 admin admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:49:51.049392 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7d604e-37bc-4b2d-885f-59e1f0783703 admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:49:51.049914 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7d604e-37bc-4b2d-885f-59e1f0783703 admin admin] PATH_INFO: `/v3/domains` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:49:51.050798 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7d604e-37bc-4b2d-885f-59e1f0783703 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86828) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:49:51.051115 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7d604e-37bc-4b2d-885f-59e1f0783703 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:49:51.052750 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7d604e-37bc-4b2d-885f-59e1f0783703 admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:49:51.067903 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1124/2272] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Jul 27 10:49:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:49:51.072245 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80075d95-47ef-4f52-905a-45176d001453 None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:49:51.084840 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80075d95-47ef-4f52-905a-45176d001453 None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:49:51.097598 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a879f349-94e1-4f93-a194-77357080271f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:49:51.097598 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a879f349-94e1-4f93-a194-77357080271f admin admin] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:49:51.097598 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a879f349-94e1-4f93-a194-77357080271f admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:49:51.097598 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a879f349-94e1-4f93-a194-77357080271f admin admin] PATH_INFO: `/v3/projects` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:49:51.098567 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a879f349-94e1-4f93-a194-77357080271f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:49:51.098567 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a879f349-94e1-4f93-a194-77357080271f admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:49:51.127916 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1149/2273] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Jul 27 10:49:51 2022] POST /identity/v3/projects => generated 383 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 10:49:51.132766 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7d604e-37bc-4b2d-885f-59e1f0783703 admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:49:51.144484 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7d604e-37bc-4b2d-885f-59e1f0783703 admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:49:51.211933 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764aac52-5283-4ae2-b44d-4052e453ad95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:49:51.212659 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764aac52-5283-4ae2-b44d-4052e453ad95 admin admin] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:49:51.212942 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764aac52-5283-4ae2-b44d-4052e453ad95 admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:49:51.213210 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764aac52-5283-4ae2-b44d-4052e453ad95 admin admin] PATH_INFO: `/v3/users` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:49:51.213879 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764aac52-5283-4ae2-b44d-4052e453ad95 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:49:51.216449 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764aac52-5283-4ae2-b44d-4052e453ad95 admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:49:51.237319 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1125/2274] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Jul 27 10:49:51 2022] POST /identity/v3/users => generated 292 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 10:49:51.241529 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a879f349-94e1-4f93-a194-77357080271f admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:49:51.252939 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a879f349-94e1-4f93-a194-77357080271f admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:49:51.257492 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77dfc510-45fa-4f7c-9132-772e0a652c0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:49:51.258092 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dfc510-45fa-4f7c-9132-772e0a652c0f admin admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:49:51.258373 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dfc510-45fa-4f7c-9132-772e0a652c0f admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:49:51.258670 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dfc510-45fa-4f7c-9132-772e0a652c0f admin admin] PATH_INFO: `/v3/roles` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:49:51.259363 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77dfc510-45fa-4f7c-9132-772e0a652c0f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:49:51.260953 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77dfc510-45fa-4f7c-9132-772e0a652c0f admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:49:51.271154 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1150/2275] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 10:49:51 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 10:49:51.275005 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764aac52-5283-4ae2-b44d-4052e453ad95 admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:49:51.285515 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764aac52-5283-4ae2-b44d-4052e453ad95 admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:49:51.290161 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65562f1e-cf4f-4bca-a203-cf2101a58d9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:49:51.290806 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65562f1e-cf4f-4bca-a203-cf2101a58d9d admin admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:49:51.291092 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65562f1e-cf4f-4bca-a203-cf2101a58d9d admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:49:51.291371 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65562f1e-cf4f-4bca-a203-cf2101a58d9d admin admin] PATH_INFO: `/v3/roles` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:49:51.292107 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65562f1e-cf4f-4bca-a203-cf2101a58d9d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:49:51.293698 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65562f1e-cf4f-4bca-a203-cf2101a58d9d admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:49:51.304025 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1126/2276] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 10:49:51 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 10:49:51.308032 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77dfc510-45fa-4f7c-9132-772e0a652c0f admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:49:51.318047 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77dfc510-45fa-4f7c-9132-772e0a652c0f admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:49:51.323152 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06190f3c-75e3-47e3-8d7a-e030abfd7e1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:49:51.324339 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06190f3c-75e3-47e3-8d7a-e030abfd7e1e admin admin] REQUEST_METHOD: `PUT` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:49:51.324622 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06190f3c-75e3-47e3-8d7a-e030abfd7e1e admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:49:51.324895 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06190f3c-75e3-47e3-8d7a-e030abfd7e1e admin admin] PATH_INFO: `/v3/projects/b961e9565f174504a1f8d4b36dc73f50/users/a90214be44404d88b8814bed43160411/roles/4c9fc0b3baed441da2126be41cc6b04a` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:49:51.340877 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06190f3c-75e3-47e3-8d7a-e030abfd7e1e admin admin] RBAC: Authorizing `identity:create_grant(project_id=b961e9565f174504a1f8d4b36dc73f50, user_id=a90214be44404d88b8814bed43160411, role_id=4c9fc0b3baed441da2126be41cc6b04a)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:49:51.343549 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06190f3c-75e3-47e3-8d7a-e030abfd7e1e admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:49:51.356091 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1151/2277] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 10:49:51 2022] PUT /identity/v3/projects/b961e9565f174504a1f8d4b36dc73f50/users/a90214be44404d88b8814bed43160411/roles/4c9fc0b3baed441da2126be41cc6b04a => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:49:51.360877 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d7b824-ab42-4e77-a251-6482e4cdead1 None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:49:51.361165 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d7b824-ab42-4e77-a251-6482e4cdead1 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:49:51.361441 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d7b824-ab42-4e77-a251-6482e4cdead1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:49:51.381450 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-b3d7b824-ab42-4e77-a251-6482e4cdead1 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:49:51.382689 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-b3d7b824-ab42-4e77-a251-6482e4cdead1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:49:51.391231 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:49:51.442059 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1127/2278] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:49:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 10:49:51.452646 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06190f3c-75e3-47e3-8d7a-e030abfd7e1e admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:49:51.454440 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-06190f3c-75e3-47e3-8d7a-e030abfd7e1e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:49:51.513589 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06190f3c-75e3-47e3-8d7a-e030abfd7e1e admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:49:51.517648 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0879f0-ef86-4f3e-a4a8-b2a47f462271 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7922d0a1a7c4e808d0b2a80dca0da40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:49:51.518273 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0879f0-ef86-4f3e-a4a8-b2a47f462271 service neutron] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:49:51.518624 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0879f0-ef86-4f3e-a4a8-b2a47f462271 service neutron] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:49:51.518914 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0879f0-ef86-4f3e-a4a8-b2a47f462271 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:49:51.528092 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0879f0-ef86-4f3e-a4a8-b2a47f462271 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:49:51.529645 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0879f0-ef86-4f3e-a4a8-b2a47f462271 service neutron] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:49:51.539084 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1152/2279] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:49:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 10:49:51.965459 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d7b824-ab42-4e77-a251-6482e4cdead1 None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:49:51.967309 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-b3d7b824-ab42-4e77-a251-6482e4cdead1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:49:51.982869 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d7b824-ab42-4e77-a251-6482e4cdead1 None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:49:51.988120 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f5c94e-ae6f-413f-b5bd-ddc5468572ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:49:51.988763 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f5c94e-ae6f-413f-b5bd-ddc5468572ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:49:51.989047 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f5c94e-ae6f-413f-b5bd-ddc5468572ee admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:49:51.989460 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f5c94e-ae6f-413f-b5bd-ddc5468572ee admin admin] PATH_INFO: `/v3/users/ef68584396384a7c9ee9962914a171da` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:49:51.990669 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f5c94e-ae6f-413f-b5bd-ddc5468572ee admin admin] RBAC: Authorizing `identity:delete_user(user_id=ef68584396384a7c9ee9962914a171da)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:49:51.992587 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f5c94e-ae6f-413f-b5bd-ddc5468572ee admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:49:52.051066 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-98f5c94e-ae6f-413f-b5bd-ddc5468572ee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ef68584396384a7c9ee9962914a171da'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:49:52.061277 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-98f5c94e-ae6f-413f-b5bd-ddc5468572ee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ef68584396384a7c9ee9962914a171da'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:49:52.070560 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-98f5c94e-ae6f-413f-b5bd-ddc5468572ee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ef68584396384a7c9ee9962914a171da'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:49:52.071550 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-98f5c94e-ae6f-413f-b5bd-ddc5468572ee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ef68584396384a7c9ee9962914a171da'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:49:52.072259 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-98f5c94e-ae6f-413f-b5bd-ddc5468572ee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ef68584396384a7c9ee9962914a171da'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:49:52.090792 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1128/2280] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Jul 27 10:49:51 2022] DELETE /identity/v3/users/ef68584396384a7c9ee9962914a171da => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 10:49:52.095247 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a0aeab-85b4-4432-a1ed-ac2d1144c428 None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:49:52.095538 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a0aeab-85b4-4432-a1ed-ac2d1144c428 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:49:52.095816 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a0aeab-85b4-4432-a1ed-ac2d1144c428 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:49:52.124380 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-78a0aeab-85b4-4432-a1ed-ac2d1144c428 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:49:52.125610 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-78a0aeab-85b4-4432-a1ed-ac2d1144c428 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:49:52.227258 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1153/2281] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:49:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:49:52.241996 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f5c94e-ae6f-413f-b5bd-ddc5468572ee admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:49:52.247602 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-98f5c94e-ae6f-413f-b5bd-ddc5468572ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:49:52.356619 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f5c94e-ae6f-413f-b5bd-ddc5468572ee admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:49:52.361114 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d5b779-0f82-4e9c-bb6d-377007dbcc98 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7922d0a1a7c4e808d0b2a80dca0da40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:49:52.361925 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d5b779-0f82-4e9c-bb6d-377007dbcc98 service neutron] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:49:52.362391 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d5b779-0f82-4e9c-bb6d-377007dbcc98 service neutron] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:49:52.362749 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d5b779-0f82-4e9c-bb6d-377007dbcc98 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:49:52.374739 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d5b779-0f82-4e9c-bb6d-377007dbcc98 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:49:52.377246 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d5b779-0f82-4e9c-bb6d-377007dbcc98 service neutron] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:49:52.394857 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1129/2282] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:49:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 10:49:52.759025 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a0aeab-85b4-4432-a1ed-ac2d1144c428 None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:49:52.760964 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-78a0aeab-85b4-4432-a1ed-ac2d1144c428 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:49:52.784278 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a0aeab-85b4-4432-a1ed-ac2d1144c428 None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:49:52.792973 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59db660c-eb01-4bae-a1c2-c5e6ecc8f310 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:49:52.793657 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59db660c-eb01-4bae-a1c2-c5e6ecc8f310 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:49:52.793929 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59db660c-eb01-4bae-a1c2-c5e6ecc8f310 admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:49:52.794406 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59db660c-eb01-4bae-a1c2-c5e6ecc8f310 admin admin] PATH_INFO: `/v3/projects/543367ebb0cc4760a1cad402d79e8fb5` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:49:52.796634 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59db660c-eb01-4bae-a1c2-c5e6ecc8f310 admin admin] RBAC: Authorizing `identity:delete_project(project_id=543367ebb0cc4760a1cad402d79e8fb5)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:49:52.801771 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59db660c-eb01-4bae-a1c2-c5e6ecc8f310 admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:49:52.821273 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:DELETE=4,keystone:SELECT=93,keystone:INSERT=3 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:49:52.903060 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-59db660c-eb01-4bae-a1c2-c5e6ecc8f310 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '543367ebb0cc4760a1cad402d79e8fb5', 'request_id': 'req-59db660c-eb01-4bae-a1c2-c5e6ecc8f310'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:49:52.912161 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-59db660c-eb01-4bae-a1c2-c5e6ecc8f310 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '543367ebb0cc4760a1cad402d79e8fb5', 'request_id': 'req-59db660c-eb01-4bae-a1c2-c5e6ecc8f310'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:49:52.923408 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-59db660c-eb01-4bae-a1c2-c5e6ecc8f310 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '543367ebb0cc4760a1cad402d79e8fb5', 'request_id': 'req-59db660c-eb01-4bae-a1c2-c5e6ecc8f310'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:49:52.949270 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-59db660c-eb01-4bae-a1c2-c5e6ecc8f310 admin admin] The token cache is being invalidate because project 543367ebb0cc4760a1cad402d79e8fb5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86827) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 10:49:52.949477 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-59db660c-eb01-4bae-a1c2-c5e6ecc8f310 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:49:52.950208 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-59db660c-eb01-4bae-a1c2-c5e6ecc8f310 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:49:52.962676 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1154/2283] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Jul 27 10:49:52 2022] DELETE /identity/v3/projects/543367ebb0cc4760a1cad402d79e8fb5 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:49:54.670852 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d5b779-0f82-4e9c-bb6d-377007dbcc98 service neutron] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:49:54.672974 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-36d5b779-0f82-4e9c-bb6d-377007dbcc98 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:49:54.690826 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d5b779-0f82-4e9c-bb6d-377007dbcc98 service neutron] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:49:54.725050 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:49:54.771664 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2437b45-df5e-41b1-8798-01f36ef64502 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:49:54.772376 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2437b45-df5e-41b1-8798-01f36ef64502 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:49:54.772671 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2437b45-df5e-41b1-8798-01f36ef64502 admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:49:54.772956 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2437b45-df5e-41b1-8798-01f36ef64502 admin admin] PATH_INFO: `/v3/users/dc28f7f7de764e91ade03830248a4e21` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:49:54.774262 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2437b45-df5e-41b1-8798-01f36ef64502 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dc28f7f7de764e91ade03830248a4e21)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:49:54.789368 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2437b45-df5e-41b1-8798-01f36ef64502 admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:49:54.843624 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-f2437b45-df5e-41b1-8798-01f36ef64502 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dc28f7f7de764e91ade03830248a4e21'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:49:54.861036 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-f2437b45-df5e-41b1-8798-01f36ef64502 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dc28f7f7de764e91ade03830248a4e21'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:49:54.872703 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-f2437b45-df5e-41b1-8798-01f36ef64502 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dc28f7f7de764e91ade03830248a4e21'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:49:54.874501 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-f2437b45-df5e-41b1-8798-01f36ef64502 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dc28f7f7de764e91ade03830248a4e21'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:49:54.875415 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-f2437b45-df5e-41b1-8798-01f36ef64502 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dc28f7f7de764e91ade03830248a4e21'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:49:54.918363 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1130/2284] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Jul 27 10:49:54 2022] DELETE /identity/v3/users/dc28f7f7de764e91ade03830248a4e21 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:49:54.923282 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03b5d5d-01eb-4f10-97f0-e9af408a2946 None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:49:54.924314 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03b5d5d-01eb-4f10-97f0-e9af408a2946 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:49:54.924723 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03b5d5d-01eb-4f10-97f0-e9af408a2946 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:49:54.998855 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-d03b5d5d-01eb-4f10-97f0-e9af408a2946 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:49:55.006137 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-d03b5d5d-01eb-4f10-97f0-e9af408a2946 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:49:55.160534 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1155/2285] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:49:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:49:55.174620 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2437b45-df5e-41b1-8798-01f36ef64502 admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:49:55.176112 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-f2437b45-df5e-41b1-8798-01f36ef64502 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:49:55.333074 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2437b45-df5e-41b1-8798-01f36ef64502 admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:49:55.338741 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fd78b2-a470-4d9a-8ef2-c9a54a4f3948 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7922d0a1a7c4e808d0b2a80dca0da40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:49:55.341063 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fd78b2-a470-4d9a-8ef2-c9a54a4f3948 service neutron] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:49:55.341063 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fd78b2-a470-4d9a-8ef2-c9a54a4f3948 service neutron] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:49:55.341063 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fd78b2-a470-4d9a-8ef2-c9a54a4f3948 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:49:55.352378 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fd78b2-a470-4d9a-8ef2-c9a54a4f3948 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:49:55.354441 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fd78b2-a470-4d9a-8ef2-c9a54a4f3948 service neutron] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:49:55.367507 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1131/2286] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:49:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 10:49:56.184167 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03b5d5d-01eb-4f10-97f0-e9af408a2946 None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:49:56.189976 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-d03b5d5d-01eb-4f10-97f0-e9af408a2946 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:49:56.201782 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03b5d5d-01eb-4f10-97f0-e9af408a2946 None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:49:56.213604 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e53f9f3-2a39-4e0d-a9bc-cc7eea261c6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:49:56.215183 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e53f9f3-2a39-4e0d-a9bc-cc7eea261c6a admin admin] REQUEST_METHOD: `DELETE` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:49:56.215470 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e53f9f3-2a39-4e0d-a9bc-cc7eea261c6a admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:49:56.215808 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e53f9f3-2a39-4e0d-a9bc-cc7eea261c6a admin admin] PATH_INFO: `/v3/projects/a99c3d709a5a475aa49411fd80e616e5` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:49:56.218206 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e53f9f3-2a39-4e0d-a9bc-cc7eea261c6a admin admin] RBAC: Authorizing `identity:delete_project(project_id=a99c3d709a5a475aa49411fd80e616e5)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:49:56.223604 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e53f9f3-2a39-4e0d-a9bc-cc7eea261c6a admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:49:56.314578 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-3e53f9f3-2a39-4e0d-a9bc-cc7eea261c6a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a99c3d709a5a475aa49411fd80e616e5', 'request_id': 'req-3e53f9f3-2a39-4e0d-a9bc-cc7eea261c6a'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:49:56.371697 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-3e53f9f3-2a39-4e0d-a9bc-cc7eea261c6a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a99c3d709a5a475aa49411fd80e616e5', 'request_id': 'req-3e53f9f3-2a39-4e0d-a9bc-cc7eea261c6a'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:49:56.398255 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-3e53f9f3-2a39-4e0d-a9bc-cc7eea261c6a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a99c3d709a5a475aa49411fd80e616e5', 'request_id': 'req-3e53f9f3-2a39-4e0d-a9bc-cc7eea261c6a'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:49:56.426011 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-3e53f9f3-2a39-4e0d-a9bc-cc7eea261c6a admin admin] The token cache is being invalidate because project a99c3d709a5a475aa49411fd80e616e5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86827) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 10:49:56.426319 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-3e53f9f3-2a39-4e0d-a9bc-cc7eea261c6a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:49:56.427184 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-3e53f9f3-2a39-4e0d-a9bc-cc7eea261c6a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:49:56.433282 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1156/2287] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Jul 27 10:49:56 2022] DELETE /identity/v3/projects/a99c3d709a5a475aa49411fd80e616e5 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:49:56.772790 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d44c899-027c-4fca-bceb-534974db6795 None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:49:56.773260 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d44c899-027c-4fca-bceb-534974db6795 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:49:56.773553 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d44c899-027c-4fca-bceb-534974db6795 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:49:56.804773 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-5d44c899-027c-4fca-bceb-534974db6795 None None] MFA Rules not processed for user `a90214be44404d88b8814bed43160411`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:49:56.806153 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-5d44c899-027c-4fca-bceb-534974db6795 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:49:56.865920 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:49:56.964748 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1132/2288] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:49:56 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:49:56.969685 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e53f9f3-2a39-4e0d-a9bc-cc7eea261c6a admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:49:56.973284 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-3e53f9f3-2a39-4e0d-a9bc-cc7eea261c6a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:49:57.032709 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:49:57.094456 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e53f9f3-2a39-4e0d-a9bc-cc7eea261c6a admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:49:57.102047 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccce5f8-5b99-4d0d-bea0-b83a05974e78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:49:57.103061 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccce5f8-5b99-4d0d-bea0-b83a05974e78 admin admin] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:49:57.103822 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccce5f8-5b99-4d0d-bea0-b83a05974e78 admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:49:57.103822 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccce5f8-5b99-4d0d-bea0-b83a05974e78 admin admin] PATH_INFO: `/v3/projects` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:49:57.104543 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cccce5f8-5b99-4d0d-bea0-b83a05974e78 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:49:57.107299 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cccce5f8-5b99-4d0d-bea0-b83a05974e78 admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:49:57.139024 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1157/2289] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Jul 27 10:49:56 2022] POST /identity/v3/projects => generated 383 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 10:49:57.144341 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d44c899-027c-4fca-bceb-534974db6795 None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:49:57.156957 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d44c899-027c-4fca-bceb-534974db6795 None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:49:57.227134 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f41f6f-0542-47e3-a651-9ca83d4ea231 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:49:57.227785 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f41f6f-0542-47e3-a651-9ca83d4ea231 admin admin] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:49:57.228070 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f41f6f-0542-47e3-a651-9ca83d4ea231 admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:49:57.228346 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f41f6f-0542-47e3-a651-9ca83d4ea231 admin admin] PATH_INFO: `/v3/users` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:49:57.229030 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f41f6f-0542-47e3-a651-9ca83d4ea231 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:49:57.231076 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f41f6f-0542-47e3-a651-9ca83d4ea231 admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:49:57.259901 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1133/2290] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Jul 27 10:49:57 2022] POST /identity/v3/users => generated 298 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 10:49:57.264389 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccce5f8-5b99-4d0d-bea0-b83a05974e78 admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:49:57.277845 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccce5f8-5b99-4d0d-bea0-b83a05974e78 admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:49:57.283774 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da5f161-621b-478c-972e-3f6c90962064 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:49:57.284604 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da5f161-621b-478c-972e-3f6c90962064 admin admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:49:57.285043 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da5f161-621b-478c-972e-3f6c90962064 admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:49:57.285514 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da5f161-621b-478c-972e-3f6c90962064 admin admin] PATH_INFO: `/v3/roles` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:49:57.286229 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da5f161-621b-478c-972e-3f6c90962064 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:49:57.288103 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da5f161-621b-478c-972e-3f6c90962064 admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:49:57.309075 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1158/2291] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 10:49:57 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 10:49:57.311000 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f41f6f-0542-47e3-a651-9ca83d4ea231 admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:49:57.322242 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f41f6f-0542-47e3-a651-9ca83d4ea231 admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:49:57.327441 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4bd7b0-a17a-4b1a-98ba-8664bad78b71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:49:57.327942 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4bd7b0-a17a-4b1a-98ba-8664bad78b71 admin admin] REQUEST_METHOD: `PUT` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:49:57.328232 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4bd7b0-a17a-4b1a-98ba-8664bad78b71 admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:49:57.328513 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4bd7b0-a17a-4b1a-98ba-8664bad78b71 admin admin] PATH_INFO: `/v3/projects/d0a98dfcb8d14e208cbd12488b4cf633/users/58f53be7efe94134bd08c64dc7da71f1/roles/21156560eb6b436bb567a40b4976f3f4` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:49:57.350806 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4bd7b0-a17a-4b1a-98ba-8664bad78b71 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d0a98dfcb8d14e208cbd12488b4cf633, user_id=58f53be7efe94134bd08c64dc7da71f1, role_id=21156560eb6b436bb567a40b4976f3f4)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:49:57.353590 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4bd7b0-a17a-4b1a-98ba-8664bad78b71 admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:49:57.375812 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1134/2292] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 10:49:57 2022] PUT /identity/v3/projects/d0a98dfcb8d14e208cbd12488b4cf633/users/58f53be7efe94134bd08c64dc7da71f1/roles/21156560eb6b436bb567a40b4976f3f4 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:49:57.380879 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da5f161-621b-478c-972e-3f6c90962064 admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:49:57.392793 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da5f161-621b-478c-972e-3f6c90962064 admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:49:57.462463 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70504af1-fd4d-4134-ab46-e065596307af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:49:57.463154 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70504af1-fd4d-4134-ab46-e065596307af admin admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:49:57.463433 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70504af1-fd4d-4134-ab46-e065596307af admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:49:57.463705 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70504af1-fd4d-4134-ab46-e065596307af admin admin] PATH_INFO: `/v3/roles` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:49:57.464442 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70504af1-fd4d-4134-ab46-e065596307af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:49:57.466057 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70504af1-fd4d-4134-ab46-e065596307af admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:49:57.478738 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1159/2293] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 10:49:57 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 10:49:57.483174 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4bd7b0-a17a-4b1a-98ba-8664bad78b71 admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:49:57.494145 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4bd7b0-a17a-4b1a-98ba-8664bad78b71 admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:49:57.501800 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815c5f82-35a7-4ae0-9aad-320e01bf8f33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:49:57.502491 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815c5f82-35a7-4ae0-9aad-320e01bf8f33 admin admin] REQUEST_METHOD: `PUT` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:49:57.502814 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815c5f82-35a7-4ae0-9aad-320e01bf8f33 admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:49:57.503092 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815c5f82-35a7-4ae0-9aad-320e01bf8f33 admin admin] PATH_INFO: `/v3/projects/d0a98dfcb8d14e208cbd12488b4cf633/users/58f53be7efe94134bd08c64dc7da71f1/roles/21156560eb6b436bb567a40b4976f3f4` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:49:57.505453 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815c5f82-35a7-4ae0-9aad-320e01bf8f33 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d0a98dfcb8d14e208cbd12488b4cf633, user_id=58f53be7efe94134bd08c64dc7da71f1, role_id=21156560eb6b436bb567a40b4976f3f4)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:49:57.508176 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815c5f82-35a7-4ae0-9aad-320e01bf8f33 admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:49:57.519604 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1135/2294] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 10:49:57 2022] PUT /identity/v3/projects/d0a98dfcb8d14e208cbd12488b4cf633/users/58f53be7efe94134bd08c64dc7da71f1/roles/21156560eb6b436bb567a40b4976f3f4 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:50:03.522737 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2117803f-f750-43a0-b13c-b07bb20286ee None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:50:03.523188 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2117803f-f750-43a0-b13c-b07bb20286ee None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:50:03.523361 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2117803f-f750-43a0-b13c-b07bb20286ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:50:03.557113 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-2117803f-f750-43a0-b13c-b07bb20286ee None None] MFA Rules not processed for user `58f53be7efe94134bd08c64dc7da71f1`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:50:03.558922 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-2117803f-f750-43a0-b13c-b07bb20286ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:50:03.604071 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815c5f82-35a7-4ae0-9aad-320e01bf8f33 admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:50:03.605846 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-815c5f82-35a7-4ae0-9aad-320e01bf8f33 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:50:03.656756 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1160/2295] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:50:03 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:50:03.669978 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2117803f-f750-43a0-b13c-b07bb20286ee None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:50:03.671750 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-2117803f-f750-43a0-b13c-b07bb20286ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:50:03.696763 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815c5f82-35a7-4ae0-9aad-320e01bf8f33 admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:50:03.718582 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796dfd8d-7595-40cb-9210-267d197ce059 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:50:03.719042 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:50:03.721504 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796dfd8d-7595-40cb-9210-267d197ce059 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:50:03.722743 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796dfd8d-7595-40cb-9210-267d197ce059 admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:50:03.726962 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796dfd8d-7595-40cb-9210-267d197ce059 admin admin] PATH_INFO: `/v3/users/43050fb9e0904edaa7fe3e48b12000ed` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:50:03.730531 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796dfd8d-7595-40cb-9210-267d197ce059 admin admin] RBAC: Authorizing `identity:delete_user(user_id=43050fb9e0904edaa7fe3e48b12000ed)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:50:03.735026 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796dfd8d-7595-40cb-9210-267d197ce059 admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:50:03.790299 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=90,keystone:INSERT=5,keystone:DELETE=5 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:50:03.796701 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2117803f-f750-43a0-b13c-b07bb20286ee None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:50:03.803893 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca13fcb-0c09-4eb5-879f-0b73c6504604 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc03c5d232654878aa1ddce15f49fdaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:50:03.804542 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca13fcb-0c09-4eb5-879f-0b73c6504604 service glance] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:50:03.804826 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca13fcb-0c09-4eb5-879f-0b73c6504604 service glance] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:50:03.805100 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca13fcb-0c09-4eb5-879f-0b73c6504604 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:50:03.811340 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-796dfd8d-7595-40cb-9210-267d197ce059 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '43050fb9e0904edaa7fe3e48b12000ed'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:50:03.821993 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-796dfd8d-7595-40cb-9210-267d197ce059 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '43050fb9e0904edaa7fe3e48b12000ed'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:50:03.832167 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-796dfd8d-7595-40cb-9210-267d197ce059 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '43050fb9e0904edaa7fe3e48b12000ed'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:50:03.833005 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-796dfd8d-7595-40cb-9210-267d197ce059 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '43050fb9e0904edaa7fe3e48b12000ed'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:50:03.833964 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-796dfd8d-7595-40cb-9210-267d197ce059 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '43050fb9e0904edaa7fe3e48b12000ed'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:50:03.859766 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1136/2296] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Jul 27 10:50:03 2022] DELETE /identity/v3/users/43050fb9e0904edaa7fe3e48b12000ed => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 10:50:03.864530 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543e7854-d867-447a-aae2-a84e27dfb7dd None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:50:03.864828 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543e7854-d867-447a-aae2-a84e27dfb7dd None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:50:03.865107 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543e7854-d867-447a-aae2-a84e27dfb7dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:50:03.907227 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-543e7854-d867-447a-aae2-a84e27dfb7dd None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:50:03.908816 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-543e7854-d867-447a-aae2-a84e27dfb7dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:50:03.925316 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca13fcb-0c09-4eb5-879f-0b73c6504604 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:50:03.927207 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca13fcb-0c09-4eb5-879f-0b73c6504604 service glance] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:50:03.939610 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1161/2297] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:50:03 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 270 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 10:50:03.989366 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1137/2298] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:50:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 10:50:03.999159 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca13fcb-0c09-4eb5-879f-0b73c6504604 service glance] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:50:04.002024 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-2ca13fcb-0c09-4eb5-879f-0b73c6504604 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:50:04.107847 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca13fcb-0c09-4eb5-879f-0b73c6504604 service glance] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:50:04.119472 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615f0ab1-66c9-457e-bdc3-03ac8b3fc22b None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:50:04.120014 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615f0ab1-66c9-457e-bdc3-03ac8b3fc22b None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:50:04.120480 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615f0ab1-66c9-457e-bdc3-03ac8b3fc22b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:50:04.148996 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-615f0ab1-66c9-457e-bdc3-03ac8b3fc22b None None] MFA Rules not processed for user `3d9fed24b4ba44b0b5961a603c4362c4`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:50:04.150480 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-615f0ab1-66c9-457e-bdc3-03ac8b3fc22b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:50:04.189438 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f672fe5b-f45e-48e5-9e09-dd6712cbacdb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7922d0a1a7c4e808d0b2a80dca0da40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:50:04.190426 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f672fe5b-f45e-48e5-9e09-dd6712cbacdb service neutron] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:50:04.190948 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f672fe5b-f45e-48e5-9e09-dd6712cbacdb service neutron] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:50:04.191436 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f672fe5b-f45e-48e5-9e09-dd6712cbacdb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:50:04.207872 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f672fe5b-f45e-48e5-9e09-dd6712cbacdb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:50:04.209489 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f672fe5b-f45e-48e5-9e09-dd6712cbacdb service neutron] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:50:04.223008 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1162/2299] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:50:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 10:50:04.250070 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1138/2300] 10.222.0.36 () {58 vars in 1009 bytes} [Wed Jul 27 10:50:04 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 10:50:04.265293 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f672fe5b-f45e-48e5-9e09-dd6712cbacdb service neutron] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:50:04.266707 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-f672fe5b-f45e-48e5-9e09-dd6712cbacdb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:50:04.307115 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:50:04.350967 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f672fe5b-f45e-48e5-9e09-dd6712cbacdb service neutron] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:50:04.356956 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e331d7e-9172-419a-a66c-e5145ee1d019 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89ac506eefcd4e42a84b44fa15fb3975', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:50:04.357498 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e331d7e-9172-419a-a66c-e5145ee1d019 service swift] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:50:04.357801 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e331d7e-9172-419a-a66c-e5145ee1d019 service swift] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:50:04.358043 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e331d7e-9172-419a-a66c-e5145ee1d019 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:50:04.366962 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e331d7e-9172-419a-a66c-e5145ee1d019 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:50:04.368687 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e331d7e-9172-419a-a66c-e5145ee1d019 service swift] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:50:04.385425 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1163/2301] 10.222.0.36 () {60 vars in 1384 bytes} [Wed Jul 27 10:50:04 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 27 10:50:04.752819 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615f0ab1-66c9-457e-bdc3-03ac8b3fc22b None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:50:04.755067 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-615f0ab1-66c9-457e-bdc3-03ac8b3fc22b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:50:04.785752 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615f0ab1-66c9-457e-bdc3-03ac8b3fc22b None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:50:04.805086 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24877bc8-5903-4e9a-aa8d-b83c174e45d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:50:04.806122 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24877bc8-5903-4e9a-aa8d-b83c174e45d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:50:04.806613 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24877bc8-5903-4e9a-aa8d-b83c174e45d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:50:04.807355 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24877bc8-5903-4e9a-aa8d-b83c174e45d8 admin admin] PATH_INFO: `/v3/projects/a18de5ec57ea4d4794f2be3cca33c097` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:50:04.809395 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24877bc8-5903-4e9a-aa8d-b83c174e45d8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a18de5ec57ea4d4794f2be3cca33c097)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:50:04.811632 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24877bc8-5903-4e9a-aa8d-b83c174e45d8 admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:50:04.907570 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-24877bc8-5903-4e9a-aa8d-b83c174e45d8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a18de5ec57ea4d4794f2be3cca33c097', 'request_id': 'req-24877bc8-5903-4e9a-aa8d-b83c174e45d8'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:50:04.910187 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:50:04.922477 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-24877bc8-5903-4e9a-aa8d-b83c174e45d8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a18de5ec57ea4d4794f2be3cca33c097', 'request_id': 'req-24877bc8-5903-4e9a-aa8d-b83c174e45d8'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:50:04.946112 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-24877bc8-5903-4e9a-aa8d-b83c174e45d8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a18de5ec57ea4d4794f2be3cca33c097', 'request_id': 'req-24877bc8-5903-4e9a-aa8d-b83c174e45d8'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:50:04.974250 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-24877bc8-5903-4e9a-aa8d-b83c174e45d8 admin admin] The token cache is being invalidate because project a18de5ec57ea4d4794f2be3cca33c097 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86828) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 10:50:04.974931 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-24877bc8-5903-4e9a-aa8d-b83c174e45d8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:50:04.975929 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-24877bc8-5903-4e9a-aa8d-b83c174e45d8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:50:04.981850 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1139/2302] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Jul 27 10:50:04 2022] DELETE /identity/v3/projects/a18de5ec57ea4d4794f2be3cca33c097 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:50:04.983192 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc87d739-3251-438c-9e3d-4f6b5a637bd1 None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:50:04.983650 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc87d739-3251-438c-9e3d-4f6b5a637bd1 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:50:04.983733 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc87d739-3251-438c-9e3d-4f6b5a637bd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:50:05.015188 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-dc87d739-3251-438c-9e3d-4f6b5a637bd1 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:50:05.016418 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-dc87d739-3251-438c-9e3d-4f6b5a637bd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:50:05.112610 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1164/2303] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:50:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 10:50:05.114703 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2a2874-6a23-4c2c-88b5-088e4b0bf032 None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:50:05.115411 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2a2874-6a23-4c2c-88b5-088e4b0bf032 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:50:05.115881 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2a2874-6a23-4c2c-88b5-088e4b0bf032 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:50:05.159363 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-4e2a2874-6a23-4c2c-88b5-088e4b0bf032 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:50:05.160961 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-4e2a2874-6a23-4c2c-88b5-088e4b0bf032 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:50:05.187116 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1140/2304] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:50:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:50:05.187325 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc87d739-3251-438c-9e3d-4f6b5a637bd1 None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:50:05.198287 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc87d739-3251-438c-9e3d-4f6b5a637bd1 None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:50:05.217112 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd19ce4-292a-4c13-88e6-b2c5467b9ee4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:50:05.217977 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd19ce4-292a-4c13-88e6-b2c5467b9ee4 admin admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:50:05.218413 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd19ce4-292a-4c13-88e6-b2c5467b9ee4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:50:05.218889 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd19ce4-292a-4c13-88e6-b2c5467b9ee4 admin admin] PATH_INFO: `/v3/domains` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:50:05.231184 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd19ce4-292a-4c13-88e6-b2c5467b9ee4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86827) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:50:05.231679 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd19ce4-292a-4c13-88e6-b2c5467b9ee4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:50:05.241343 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd19ce4-292a-4c13-88e6-b2c5467b9ee4 admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:50:05.270909 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1165/2305] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Jul 27 10:50:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 10:50:05.275348 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2a2874-6a23-4c2c-88b5-088e4b0bf032 None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:50:05.293687 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2a2874-6a23-4c2c-88b5-088e4b0bf032 None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:50:05.301247 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4b93be-ba3c-4513-be93-ab452ab04816 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:50:05.302198 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4b93be-ba3c-4513-be93-ab452ab04816 admin admin] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:50:05.302743 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4b93be-ba3c-4513-be93-ab452ab04816 admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:50:05.303404 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4b93be-ba3c-4513-be93-ab452ab04816 admin admin] PATH_INFO: `/v3/projects` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:50:05.304398 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4b93be-ba3c-4513-be93-ab452ab04816 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:50:05.308217 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4b93be-ba3c-4513-be93-ab452ab04816 admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:50:05.363208 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1141/2306] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Jul 27 10:50:05 2022] POST /identity/v3/projects => generated 383 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 10:50:05.366975 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd19ce4-292a-4c13-88e6-b2c5467b9ee4 admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:50:05.382939 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd19ce4-292a-4c13-88e6-b2c5467b9ee4 admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:50:05.548497 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38c2a6f-6ea4-42ca-a637-be41b4148097 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:50:05.549430 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38c2a6f-6ea4-42ca-a637-be41b4148097 admin admin] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:50:05.550731 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38c2a6f-6ea4-42ca-a637-be41b4148097 admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:50:05.551212 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38c2a6f-6ea4-42ca-a637-be41b4148097 admin admin] PATH_INFO: `/v3/users` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:50:05.552125 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38c2a6f-6ea4-42ca-a637-be41b4148097 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:50:05.554910 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38c2a6f-6ea4-42ca-a637-be41b4148097 admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:50:05.610170 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1166/2307] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Jul 27 10:50:05 2022] POST /identity/v3/users => generated 292 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 10:50:05.614007 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4b93be-ba3c-4513-be93-ab452ab04816 admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:50:05.629255 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4b93be-ba3c-4513-be93-ab452ab04816 admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:50:05.639278 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab95231-e123-4bf9-a5a7-bac3614c733e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:50:05.639918 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab95231-e123-4bf9-a5a7-bac3614c733e admin admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:50:05.640204 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab95231-e123-4bf9-a5a7-bac3614c733e admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:50:05.640479 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab95231-e123-4bf9-a5a7-bac3614c733e admin admin] PATH_INFO: `/v3/roles` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:50:05.641278 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab95231-e123-4bf9-a5a7-bac3614c733e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:50:05.643130 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab95231-e123-4bf9-a5a7-bac3614c733e admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:50:05.662150 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1142/2308] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 10:50:05 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 10:50:05.663426 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38c2a6f-6ea4-42ca-a637-be41b4148097 admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:50:05.685896 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38c2a6f-6ea4-42ca-a637-be41b4148097 admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:50:05.693863 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3645a99-9fa1-470d-a687-671522e19295 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:50:05.694870 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3645a99-9fa1-470d-a687-671522e19295 admin admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:50:05.695559 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3645a99-9fa1-470d-a687-671522e19295 admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:50:05.695944 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3645a99-9fa1-470d-a687-671522e19295 admin admin] PATH_INFO: `/v3/roles` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:50:05.709769 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3645a99-9fa1-470d-a687-671522e19295 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:50:05.712617 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3645a99-9fa1-470d-a687-671522e19295 admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:50:05.735230 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1167/2309] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 10:50:05 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 10:50:05.740066 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab95231-e123-4bf9-a5a7-bac3614c733e admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:50:05.770059 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab95231-e123-4bf9-a5a7-bac3614c733e admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:50:05.780543 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fd2000-3cec-4f23-ab77-d12b58067b05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:50:05.781396 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fd2000-3cec-4f23-ab77-d12b58067b05 admin admin] REQUEST_METHOD: `PUT` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:50:05.781778 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fd2000-3cec-4f23-ab77-d12b58067b05 admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:50:05.782433 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fd2000-3cec-4f23-ab77-d12b58067b05 admin admin] PATH_INFO: `/v3/projects/01d0ee48032146aaa8842b9ad02ff30a/users/1c8b9cc69c1142d780bc707e777d1bad/roles/4c9fc0b3baed441da2126be41cc6b04a` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:50:05.806555 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44fd2000-3cec-4f23-ab77-d12b58067b05 admin admin] RBAC: Authorizing `identity:create_grant(project_id=01d0ee48032146aaa8842b9ad02ff30a, user_id=1c8b9cc69c1142d780bc707e777d1bad, role_id=4c9fc0b3baed441da2126be41cc6b04a)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:50:05.815396 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44fd2000-3cec-4f23-ab77-d12b58067b05 admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:50:05.844253 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1143/2310] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 10:50:05 2022] PUT /identity/v3/projects/01d0ee48032146aaa8842b9ad02ff30a/users/1c8b9cc69c1142d780bc707e777d1bad/roles/4c9fc0b3baed441da2126be41cc6b04a => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:50:05.864140 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c3b72a-b177-4f2e-80d5-1c5646e5d605 None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:50:05.864805 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c3b72a-b177-4f2e-80d5-1c5646e5d605 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:50:05.865884 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c3b72a-b177-4f2e-80d5-1c5646e5d605 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:50:05.918045 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-22c3b72a-b177-4f2e-80d5-1c5646e5d605 None None] MFA Rules not processed for user `1c8b9cc69c1142d780bc707e777d1bad`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:50:05.919336 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-22c3b72a-b177-4f2e-80d5-1c5646e5d605 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:50:05.937286 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:50:06.017421 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1168/2311] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:50:05 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:50:06.029680 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fd2000-3cec-4f23-ab77-d12b58067b05 admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:50:06.032170 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-44fd2000-3cec-4f23-ab77-d12b58067b05 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:50:06.105258 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fd2000-3cec-4f23-ab77-d12b58067b05 admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:50:06.109450 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac8d3af-f458-49b9-9ab7-fd6a7e24d063 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '864d35f268cf45148756109759bb2743', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:50:06.110118 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac8d3af-f458-49b9-9ab7-fd6a7e24d063 service cinder] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:50:06.110526 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac8d3af-f458-49b9-9ab7-fd6a7e24d063 service cinder] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:50:06.111103 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac8d3af-f458-49b9-9ab7-fd6a7e24d063 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:50:06.120853 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac8d3af-f458-49b9-9ab7-fd6a7e24d063 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:50:06.122487 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac8d3af-f458-49b9-9ab7-fd6a7e24d063 service cinder] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:50:06.132840 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1144/2312] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:50:06 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 10:50:06.222181 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c262e3a-cbde-4c1c-b133-23df5b977b26 None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:50:06.222869 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c262e3a-cbde-4c1c-b133-23df5b977b26 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:50:06.223309 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c262e3a-cbde-4c1c-b133-23df5b977b26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:50:06.257843 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-2c262e3a-cbde-4c1c-b133-23df5b977b26 None None] MFA Rules not processed for user `5ae9a5696bdc48778835da7a311f20c7`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:50:06.261073 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-2c262e3a-cbde-4c1c-b133-23df5b977b26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:50:06.361711 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1169/2313] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:50:06 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:50:09.065390 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac8d3af-f458-49b9-9ab7-fd6a7e24d063 service cinder] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:50:09.202367 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac8d3af-f458-49b9-9ab7-fd6a7e24d063 service cinder] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:50:09.219998 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2692b60-dfeb-493f-9c56-980441df1174 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:50:09.220973 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2692b60-dfeb-493f-9c56-980441df1174 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:50:09.221425 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2692b60-dfeb-493f-9c56-980441df1174 admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:50:09.221867 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2692b60-dfeb-493f-9c56-980441df1174 admin admin] PATH_INFO: `/v3/users/1c8b9cc69c1142d780bc707e777d1bad` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:50:09.223298 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2692b60-dfeb-493f-9c56-980441df1174 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1c8b9cc69c1142d780bc707e777d1bad)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:50:09.225428 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2692b60-dfeb-493f-9c56-980441df1174 admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:50:09.267369 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=91,keystone:INSERT=3,keystone:DELETE=6 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:50:09.319105 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-c2692b60-dfeb-493f-9c56-980441df1174 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1c8b9cc69c1142d780bc707e777d1bad'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:50:09.337383 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-c2692b60-dfeb-493f-9c56-980441df1174 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1c8b9cc69c1142d780bc707e777d1bad'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:50:09.356738 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-c2692b60-dfeb-493f-9c56-980441df1174 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1c8b9cc69c1142d780bc707e777d1bad'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:50:09.357937 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-c2692b60-dfeb-493f-9c56-980441df1174 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1c8b9cc69c1142d780bc707e777d1bad'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:50:09.359038 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-c2692b60-dfeb-493f-9c56-980441df1174 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1c8b9cc69c1142d780bc707e777d1bad'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:50:09.389704 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1145/2314] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Jul 27 10:50:09 2022] DELETE /identity/v3/users/1c8b9cc69c1142d780bc707e777d1bad => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:50:09.394829 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f290b753-7ddb-435a-bbb1-a4063698b1ef None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:50:09.396802 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f290b753-7ddb-435a-bbb1-a4063698b1ef None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:50:09.397245 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f290b753-7ddb-435a-bbb1-a4063698b1ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:50:09.437766 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-f290b753-7ddb-435a-bbb1-a4063698b1ef None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:50:09.440192 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-f290b753-7ddb-435a-bbb1-a4063698b1ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:50:09.610639 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1170/2315] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:50:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:50:09.622139 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2692b60-dfeb-493f-9c56-980441df1174 admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:50:09.623895 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-c2692b60-dfeb-493f-9c56-980441df1174 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:50:09.786648 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2692b60-dfeb-493f-9c56-980441df1174 admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:50:09.806523 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e0a781-d054-405c-8d33-4db11ec9c0b1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7922d0a1a7c4e808d0b2a80dca0da40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:50:09.807239 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e0a781-d054-405c-8d33-4db11ec9c0b1 service neutron] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:50:09.807503 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e0a781-d054-405c-8d33-4db11ec9c0b1 service neutron] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:50:09.807824 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e0a781-d054-405c-8d33-4db11ec9c0b1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:50:09.869170 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e0a781-d054-405c-8d33-4db11ec9c0b1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:50:09.873388 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e0a781-d054-405c-8d33-4db11ec9c0b1 service neutron] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:50:09.911546 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1146/2316] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:50:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 10:50:10.655608 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f290b753-7ddb-435a-bbb1-a4063698b1ef None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:50:10.658969 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-f290b753-7ddb-435a-bbb1-a4063698b1ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:50:10.695728 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f290b753-7ddb-435a-bbb1-a4063698b1ef None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:50:10.704757 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed23148-2e2e-439f-9d3f-8cb0221c51c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:50:10.705423 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed23148-2e2e-439f-9d3f-8cb0221c51c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:50:10.705754 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed23148-2e2e-439f-9d3f-8cb0221c51c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:50:10.706038 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed23148-2e2e-439f-9d3f-8cb0221c51c8 admin admin] PATH_INFO: `/v3/projects/01d0ee48032146aaa8842b9ad02ff30a` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:50:10.707251 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed23148-2e2e-439f-9d3f-8cb0221c51c8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=01d0ee48032146aaa8842b9ad02ff30a)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:50:10.711021 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed23148-2e2e-439f-9d3f-8cb0221c51c8 admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:50:10.804131 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-aed23148-2e2e-439f-9d3f-8cb0221c51c8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '01d0ee48032146aaa8842b9ad02ff30a', 'request_id': 'req-aed23148-2e2e-439f-9d3f-8cb0221c51c8'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:50:10.816185 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-aed23148-2e2e-439f-9d3f-8cb0221c51c8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '01d0ee48032146aaa8842b9ad02ff30a', 'request_id': 'req-aed23148-2e2e-439f-9d3f-8cb0221c51c8'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:50:10.818372 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:50:10.827894 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-aed23148-2e2e-439f-9d3f-8cb0221c51c8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '01d0ee48032146aaa8842b9ad02ff30a', 'request_id': 'req-aed23148-2e2e-439f-9d3f-8cb0221c51c8'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:50:10.857988 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-aed23148-2e2e-439f-9d3f-8cb0221c51c8 admin admin] The token cache is being invalidate because project 01d0ee48032146aaa8842b9ad02ff30a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86827) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 10:50:10.858285 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-aed23148-2e2e-439f-9d3f-8cb0221c51c8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:50:10.862219 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-aed23148-2e2e-439f-9d3f-8cb0221c51c8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:50:10.864329 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1171/2317] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Jul 27 10:50:10 2022] DELETE /identity/v3/projects/01d0ee48032146aaa8842b9ad02ff30a => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:50:10.869242 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059f9af3-4363-43b1-925f-cb80cbe3cb46 None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:50:10.870250 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059f9af3-4363-43b1-925f-cb80cbe3cb46 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:50:10.870566 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059f9af3-4363-43b1-925f-cb80cbe3cb46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:50:10.913460 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-059f9af3-4363-43b1-925f-cb80cbe3cb46 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:50:10.915453 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-059f9af3-4363-43b1-925f-cb80cbe3cb46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:50:11.017058 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f37b8f7-1fcc-485f-a982-f4b20366aa85 None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:50:11.017289 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1147/2318] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:50:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:50:11.017411 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f37b8f7-1fcc-485f-a982-f4b20366aa85 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:50:11.017570 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f37b8f7-1fcc-485f-a982-f4b20366aa85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:50:11.054478 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-9f37b8f7-1fcc-485f-a982-f4b20366aa85 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:50:11.055905 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-9f37b8f7-1fcc-485f-a982-f4b20366aa85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:50:11.068835 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1172/2319] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:50:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 10:50:11.070073 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059f9af3-4363-43b1-925f-cb80cbe3cb46 None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:50:11.084506 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059f9af3-4363-43b1-925f-cb80cbe3cb46 None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:50:11.107140 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3ff44a-8d29-43c2-ab55-ef2c512d736d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:50:11.107808 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3ff44a-8d29-43c2-ab55-ef2c512d736d admin admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:50:11.108094 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3ff44a-8d29-43c2-ab55-ef2c512d736d admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:50:11.108373 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3ff44a-8d29-43c2-ab55-ef2c512d736d admin admin] PATH_INFO: `/v3/domains` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:50:11.109099 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3ff44a-8d29-43c2-ab55-ef2c512d736d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86828) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:50:11.109408 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3ff44a-8d29-43c2-ab55-ef2c512d736d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:50:11.122289 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3ff44a-8d29-43c2-ab55-ef2c512d736d admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:50:11.165728 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f37b8f7-1fcc-485f-a982-f4b20366aa85 None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:50:11.169372 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1148/2320] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Jul 27 10:50:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:50:11.187270 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f37b8f7-1fcc-485f-a982-f4b20366aa85 None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:50:11.205347 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabcf014-369e-493d-8930-838680d1fb3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:50:11.205998 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabcf014-369e-493d-8930-838680d1fb3b admin admin] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:50:11.206278 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabcf014-369e-493d-8930-838680d1fb3b admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:50:11.206549 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabcf014-369e-493d-8930-838680d1fb3b admin admin] PATH_INFO: `/v3/projects` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:50:11.207339 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabcf014-369e-493d-8930-838680d1fb3b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:50:11.209330 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabcf014-369e-493d-8930-838680d1fb3b admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:50:11.305023 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1173/2321] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Jul 27 10:50:11 2022] POST /identity/v3/projects => generated 365 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 10:50:11.306775 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3ff44a-8d29-43c2-ab55-ef2c512d736d admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:50:11.341181 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3ff44a-8d29-43c2-ab55-ef2c512d736d admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:50:11.398799 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:50:11.516011 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34555406-6a1f-42ed-82bf-b647befa1bcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:50:11.516659 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34555406-6a1f-42ed-82bf-b647befa1bcd admin admin] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:50:11.518042 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34555406-6a1f-42ed-82bf-b647befa1bcd admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:50:11.518325 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34555406-6a1f-42ed-82bf-b647befa1bcd admin admin] PATH_INFO: `/v3/users` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:50:11.519063 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34555406-6a1f-42ed-82bf-b647befa1bcd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:50:11.523400 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34555406-6a1f-42ed-82bf-b647befa1bcd admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:50:11.580473 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1149/2322] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Jul 27 10:50:11 2022] POST /identity/v3/users => generated 283 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 10:50:11.584717 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabcf014-369e-493d-8930-838680d1fb3b admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:50:11.610367 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabcf014-369e-493d-8930-838680d1fb3b admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:50:11.627071 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bd8a2e-d6b5-4eff-8909-083d9e1e86ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:50:11.634496 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bd8a2e-d6b5-4eff-8909-083d9e1e86ce admin admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:50:11.634920 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bd8a2e-d6b5-4eff-8909-083d9e1e86ce admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:50:11.635299 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bd8a2e-d6b5-4eff-8909-083d9e1e86ce admin admin] PATH_INFO: `/v3/roles` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:50:11.637391 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2bd8a2e-d6b5-4eff-8909-083d9e1e86ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:50:11.639189 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2bd8a2e-d6b5-4eff-8909-083d9e1e86ce admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:50:11.681350 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1174/2323] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 10:50:11 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 10:50:11.685886 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34555406-6a1f-42ed-82bf-b647befa1bcd admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:50:11.703998 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34555406-6a1f-42ed-82bf-b647befa1bcd admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:50:11.712989 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecf5290-9f53-4913-bc02-bb8fa5f32ee3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:50:11.716072 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecf5290-9f53-4913-bc02-bb8fa5f32ee3 admin admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:50:11.717407 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecf5290-9f53-4913-bc02-bb8fa5f32ee3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:50:11.717800 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecf5290-9f53-4913-bc02-bb8fa5f32ee3 admin admin] PATH_INFO: `/v3/roles` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:50:11.719171 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ecf5290-9f53-4913-bc02-bb8fa5f32ee3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:50:11.722087 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ecf5290-9f53-4913-bc02-bb8fa5f32ee3 admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:50:11.738654 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1150/2324] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 10:50:11 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 10:50:11.743729 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bd8a2e-d6b5-4eff-8909-083d9e1e86ce admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:50:11.757970 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bd8a2e-d6b5-4eff-8909-083d9e1e86ce admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:50:11.766363 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29befc0-5e23-4a87-8c11-9885e8aec04b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:50:11.770400 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29befc0-5e23-4a87-8c11-9885e8aec04b admin admin] REQUEST_METHOD: `PUT` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:50:11.770824 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29befc0-5e23-4a87-8c11-9885e8aec04b admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:50:11.771205 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29befc0-5e23-4a87-8c11-9885e8aec04b admin admin] PATH_INFO: `/v3/projects/efe888e5e5834b49917eb91b4ad89d89/users/191953f8df1948bcaec9ac03d9e3057c/roles/4c9fc0b3baed441da2126be41cc6b04a` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:50:11.806561 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29befc0-5e23-4a87-8c11-9885e8aec04b admin admin] RBAC: Authorizing `identity:create_grant(project_id=efe888e5e5834b49917eb91b4ad89d89, user_id=191953f8df1948bcaec9ac03d9e3057c, role_id=4c9fc0b3baed441da2126be41cc6b04a)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:50:11.812903 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29befc0-5e23-4a87-8c11-9885e8aec04b admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:50:11.837661 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1175/2325] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 10:50:11 2022] PUT /identity/v3/projects/efe888e5e5834b49917eb91b4ad89d89/users/191953f8df1948bcaec9ac03d9e3057c/roles/4c9fc0b3baed441da2126be41cc6b04a => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:50:11.850580 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee7d9ab-9b71-452d-a5c8-710e8bb8a773 None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:50:11.850900 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee7d9ab-9b71-452d-a5c8-710e8bb8a773 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:50:11.851185 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee7d9ab-9b71-452d-a5c8-710e8bb8a773 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:50:11.897097 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-5ee7d9ab-9b71-452d-a5c8-710e8bb8a773 None None] MFA Rules not processed for user `191953f8df1948bcaec9ac03d9e3057c`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:50:11.899200 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-5ee7d9ab-9b71-452d-a5c8-710e8bb8a773 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:50:11.997685 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1151/2326] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:50:11 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:50:12.006440 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29befc0-5e23-4a87-8c11-9885e8aec04b admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:50:12.011383 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-f29befc0-5e23-4a87-8c11-9885e8aec04b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:50:12.079038 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29befc0-5e23-4a87-8c11-9885e8aec04b admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:50:12.082793 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec1dd55-f3d2-41c2-a937-e3a944268910 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '864d35f268cf45148756109759bb2743', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:50:12.083416 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec1dd55-f3d2-41c2-a937-e3a944268910 service cinder] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:50:12.083700 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec1dd55-f3d2-41c2-a937-e3a944268910 service cinder] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:50:12.084489 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec1dd55-f3d2-41c2-a937-e3a944268910 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:50:12.095920 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec1dd55-f3d2-41c2-a937-e3a944268910 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:50:12.097360 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec1dd55-f3d2-41c2-a937-e3a944268910 service cinder] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:50:12.111715 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1176/2327] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:50:12 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 10:50:16.465357 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c32527-7eaa-4866-8f1a-d87b3b4c3959 None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:50:16.465915 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c32527-7eaa-4866-8f1a-d87b3b4c3959 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:50:16.465915 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c32527-7eaa-4866-8f1a-d87b3b4c3959 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:50:16.485113 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-47c32527-7eaa-4866-8f1a-d87b3b4c3959 None None] MFA Rules not processed for user `a90214be44404d88b8814bed43160411`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:50:16.486337 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-47c32527-7eaa-4866-8f1a-d87b3b4c3959 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:50:16.545725 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1152/2328] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:50:16 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:50:18.597834 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a33e667-db1a-4759-8e62-9407284d4292 None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:50:18.598352 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a33e667-db1a-4759-8e62-9407284d4292 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:50:18.598434 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a33e667-db1a-4759-8e62-9407284d4292 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:50:18.618194 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-2a33e667-db1a-4759-8e62-9407284d4292 None None] MFA Rules not processed for user `3d9fed24b4ba44b0b5961a603c4362c4`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:50:18.619614 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-2a33e667-db1a-4759-8e62-9407284d4292 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:50:18.680446 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1177/2329] 10.222.0.36 () {58 vars in 1009 bytes} [Wed Jul 27 10:50:18 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:50:18.694955 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c32527-7eaa-4866-8f1a-d87b3b4c3959 None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:50:18.696446 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-47c32527-7eaa-4866-8f1a-d87b3b4c3959 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:50:18.747535 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:50:18.773109 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c32527-7eaa-4866-8f1a-d87b3b4c3959 None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:50:18.779281 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd852584-2723-4784-9062-1f8bd7d1b432 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89ac506eefcd4e42a84b44fa15fb3975', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:50:18.780344 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd852584-2723-4784-9062-1f8bd7d1b432 service swift] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:50:18.780780 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd852584-2723-4784-9062-1f8bd7d1b432 service swift] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:50:18.781198 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd852584-2723-4784-9062-1f8bd7d1b432 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:50:18.791927 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd852584-2723-4784-9062-1f8bd7d1b432 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:50:18.794084 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd852584-2723-4784-9062-1f8bd7d1b432 service swift] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:50:18.802241 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1153/2330] 10.222.0.36 () {60 vars in 1384 bytes} [Wed Jul 27 10:50:18 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 27 10:50:28.670683 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:INSERT=3,keystone:DELETE=3,keystone:SELECT=82 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:50:28.799042 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=13 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:50:48.683704 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a33e667-db1a-4759-8e62-9407284d4292 None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:50:48.685708 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-2a33e667-db1a-4759-8e62-9407284d4292 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:50:48.745037 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a33e667-db1a-4759-8e62-9407284d4292 None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:50:48.749700 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f204fa46-c452-4855-8c4d-04377b9284db service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7922d0a1a7c4e808d0b2a80dca0da40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:50:48.751025 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f204fa46-c452-4855-8c4d-04377b9284db service neutron] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:50:48.751547 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f204fa46-c452-4855-8c4d-04377b9284db service neutron] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:50:48.751869 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f204fa46-c452-4855-8c4d-04377b9284db service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:50:48.754152 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-f204fa46-c452-4855-8c4d-04377b9284db service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:50:48.820976 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f204fa46-c452-4855-8c4d-04377b9284db service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:50:48.822412 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f204fa46-c452-4855-8c4d-04377b9284db service neutron] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:50:48.834426 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1178/2331] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:50:48 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 10:50:48.891510 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd852584-2723-4784-9062-1f8bd7d1b432 service swift] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:50:48.959981 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd852584-2723-4784-9062-1f8bd7d1b432 service swift] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:50:48.965442 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1766090-e001-45c2-b735-71cecc20a29b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:50:48.966089 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1766090-e001-45c2-b735-71cecc20a29b admin admin] REQUEST_METHOD: `DELETE` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:50:48.966369 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1766090-e001-45c2-b735-71cecc20a29b admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:50:48.966669 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1766090-e001-45c2-b735-71cecc20a29b admin admin] PATH_INFO: `/v3/users/191953f8df1948bcaec9ac03d9e3057c` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:50:48.967834 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1766090-e001-45c2-b735-71cecc20a29b admin admin] RBAC: Authorizing `identity:delete_user(user_id=191953f8df1948bcaec9ac03d9e3057c)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:50:48.969793 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1766090-e001-45c2-b735-71cecc20a29b admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:50:49.022544 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-a1766090-e001-45c2-b735-71cecc20a29b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '191953f8df1948bcaec9ac03d9e3057c'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:50:49.030086 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-a1766090-e001-45c2-b735-71cecc20a29b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '191953f8df1948bcaec9ac03d9e3057c'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:50:49.037262 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-a1766090-e001-45c2-b735-71cecc20a29b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '191953f8df1948bcaec9ac03d9e3057c'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:50:49.037972 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-a1766090-e001-45c2-b735-71cecc20a29b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '191953f8df1948bcaec9ac03d9e3057c'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:50:49.038620 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-a1766090-e001-45c2-b735-71cecc20a29b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '191953f8df1948bcaec9ac03d9e3057c'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:50:49.059297 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1154/2332] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Jul 27 10:50:48 2022] DELETE /identity/v3/users/191953f8df1948bcaec9ac03d9e3057c => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 10:50:49.063828 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8bfa9f-60f5-496c-8585-ccc97d091a1b None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:50:49.064112 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8bfa9f-60f5-496c-8585-ccc97d091a1b None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:50:49.064392 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8bfa9f-60f5-496c-8585-ccc97d091a1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:50:49.083657 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-ee8bfa9f-60f5-496c-8585-ccc97d091a1b None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:50:49.085056 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-ee8bfa9f-60f5-496c-8585-ccc97d091a1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:50:49.153668 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1179/2333] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:50:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:50:49.165191 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1766090-e001-45c2-b735-71cecc20a29b admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:50:49.167325 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-a1766090-e001-45c2-b735-71cecc20a29b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:50:49.233435 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1766090-e001-45c2-b735-71cecc20a29b admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:50:49.237573 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d738d764-790f-4b9e-bc14-233b76383803 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7922d0a1a7c4e808d0b2a80dca0da40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:50:49.238205 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d738d764-790f-4b9e-bc14-233b76383803 service neutron] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:50:49.238488 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d738d764-790f-4b9e-bc14-233b76383803 service neutron] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:50:49.238796 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d738d764-790f-4b9e-bc14-233b76383803 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:50:49.248160 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d738d764-790f-4b9e-bc14-233b76383803 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:50:49.249592 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d738d764-790f-4b9e-bc14-233b76383803 service neutron] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:50:49.259679 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1155/2334] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:50:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 10:50:49.788231 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8bfa9f-60f5-496c-8585-ccc97d091a1b None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:50:49.791136 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-ee8bfa9f-60f5-496c-8585-ccc97d091a1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:50:49.802435 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8bfa9f-60f5-496c-8585-ccc97d091a1b None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:50:49.807130 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884dd198-1624-473b-8349-b5761e415ea0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:50:49.807780 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884dd198-1624-473b-8349-b5761e415ea0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:50:49.808057 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884dd198-1624-473b-8349-b5761e415ea0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:50:49.808330 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884dd198-1624-473b-8349-b5761e415ea0 admin admin] PATH_INFO: `/v3/projects/efe888e5e5834b49917eb91b4ad89d89` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:50:49.809485 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884dd198-1624-473b-8349-b5761e415ea0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=efe888e5e5834b49917eb91b4ad89d89)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:50:49.811689 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884dd198-1624-473b-8349-b5761e415ea0 admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:50:49.869756 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:50:49.876677 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-884dd198-1624-473b-8349-b5761e415ea0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'efe888e5e5834b49917eb91b4ad89d89', 'request_id': 'req-884dd198-1624-473b-8349-b5761e415ea0'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:50:49.884813 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-884dd198-1624-473b-8349-b5761e415ea0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'efe888e5e5834b49917eb91b4ad89d89', 'request_id': 'req-884dd198-1624-473b-8349-b5761e415ea0'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:50:49.895590 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-884dd198-1624-473b-8349-b5761e415ea0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'efe888e5e5834b49917eb91b4ad89d89', 'request_id': 'req-884dd198-1624-473b-8349-b5761e415ea0'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:50:49.907695 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-884dd198-1624-473b-8349-b5761e415ea0 admin admin] The token cache is being invalidate because project efe888e5e5834b49917eb91b4ad89d89 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86827) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 10:50:49.907988 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-884dd198-1624-473b-8349-b5761e415ea0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:50:49.908648 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-884dd198-1624-473b-8349-b5761e415ea0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:50:49.910198 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1180/2335] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Jul 27 10:50:49 2022] DELETE /identity/v3/projects/efe888e5e5834b49917eb91b4ad89d89 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:50:56.934869 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d738d764-790f-4b9e-bc14-233b76383803 service neutron] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:50:56.936270 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-d738d764-790f-4b9e-bc14-233b76383803 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:50:56.963187 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:50:57.013203 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d738d764-790f-4b9e-bc14-233b76383803 service neutron] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:50:57.070734 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9f2889-aae9-44a3-9369-beb5a7289b94 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16cc6179e9a04b6398fbacb40ad1c555', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:50:57.071357 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9f2889-aae9-44a3-9369-beb5a7289b94 service placement] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:50:57.071639 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9f2889-aae9-44a3-9369-beb5a7289b94 service placement] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:50:57.071915 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9f2889-aae9-44a3-9369-beb5a7289b94 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:50:57.074887 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-3e9f2889-aae9-44a3-9369-beb5a7289b94 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:50:57.086713 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9f2889-aae9-44a3-9369-beb5a7289b94 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:50:57.089068 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9f2889-aae9-44a3-9369-beb5a7289b94 service placement] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:50:57.099090 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1156/2336] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:50:56 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 10:50:59.904580 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:DELETE=4,keystone:SELECT=7,keystone:INSERT=1 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:51:04.895208 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d39373-b7a8-4891-b16f-6ec10e18f443 None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:51:04.895800 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d39373-b7a8-4891-b16f-6ec10e18f443 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:51:04.895800 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d39373-b7a8-4891-b16f-6ec10e18f443 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:51:04.915486 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-73d39373-b7a8-4891-b16f-6ec10e18f443 None None] MFA Rules not processed for user `3d9fed24b4ba44b0b5961a603c4362c4`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:51:04.917335 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-73d39373-b7a8-4891-b16f-6ec10e18f443 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:51:04.973165 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1181/2337] 10.222.0.36 () {58 vars in 1009 bytes} [Wed Jul 27 10:51:04 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:51:04.995473 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9f2889-aae9-44a3-9369-beb5a7289b94 service placement] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:51:04.996888 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-3e9f2889-aae9-44a3-9369-beb5a7289b94 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:51:05.067640 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9f2889-aae9-44a3-9369-beb5a7289b94 service placement] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:51:05.073326 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96d2850-0a9c-4087-ae84-7eb43b9f0fc9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89ac506eefcd4e42a84b44fa15fb3975', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:51:05.073944 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96d2850-0a9c-4087-ae84-7eb43b9f0fc9 service swift] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:51:05.074229 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96d2850-0a9c-4087-ae84-7eb43b9f0fc9 service swift] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:51:05.074508 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96d2850-0a9c-4087-ae84-7eb43b9f0fc9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:51:05.082451 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96d2850-0a9c-4087-ae84-7eb43b9f0fc9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:51:05.084079 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96d2850-0a9c-4087-ae84-7eb43b9f0fc9 service swift] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:51:05.092556 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1157/2338] 10.222.0.36 () {60 vars in 1384 bytes} [Wed Jul 27 10:51:04 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 27 10:51:14.966190 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=25 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:51:15.089353 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=80 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:51:18.055565 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f23122-11ec-4dd0-ac46-28c297bdcca5 None None] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:51:18.056137 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f23122-11ec-4dd0-ac46-28c297bdcca5 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:51:18.056268 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f23122-11ec-4dd0-ac46-28c297bdcca5 None None] PATH_INFO: `/` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:51:18.057606 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1182/2339] 10.222.0.36 () {54 vars in 921 bytes} [Wed Jul 27 10:51:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 10:51:18.063454 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acb45e9-3e77-483b-be35-40f34a781d8a None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:51:18.063916 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acb45e9-3e77-483b-be35-40f34a781d8a None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:51:18.063916 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acb45e9-3e77-483b-be35-40f34a781d8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:51:18.084432 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-8acb45e9-3e77-483b-be35-40f34a781d8a None None] MFA Rules not processed for user `e059d8b6c28c4fcba925d05d7c98e4ac`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:51:18.085620 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-8acb45e9-3e77-483b-be35-40f34a781d8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:51:18.155744 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1158/2340] 10.222.0.36 () {58 vars in 1018 bytes} [Wed Jul 27 10:51:18 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 10:51:18.167545 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f23122-11ec-4dd0-ac46-28c297bdcca5 None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:51:18.169363 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-c2f23122-11ec-4dd0-ac46-28c297bdcca5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:51:18.227743 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f23122-11ec-4dd0-ac46-28c297bdcca5 None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:51:18.231253 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6913e243-abf0-45d9-9eb0-802d1bb478b8 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc03c5d232654878aa1ddce15f49fdaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:51:18.231859 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6913e243-abf0-45d9-9eb0-802d1bb478b8 service glance] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:51:18.232143 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6913e243-abf0-45d9-9eb0-802d1bb478b8 service glance] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:51:18.232415 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6913e243-abf0-45d9-9eb0-802d1bb478b8 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:51:18.242402 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6913e243-abf0-45d9-9eb0-802d1bb478b8 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:51:18.243849 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6913e243-abf0-45d9-9eb0-802d1bb478b8 service glance] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:51:18.255051 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1183/2341] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:51:18 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 10:51:18.323144 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b26a324-5474-4be9-b0a8-93a61eca2f65 None None] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:51:18.323342 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b26a324-5474-4be9-b0a8-93a61eca2f65 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:51:18.323623 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b26a324-5474-4be9-b0a8-93a61eca2f65 None None] PATH_INFO: `/` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:51:18.324997 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1159/2342] 10.222.0.36 () {54 vars in 921 bytes} [Wed Jul 27 10:51:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 10:51:18.329958 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5b90ec-cf61-44ab-87ac-6ed105afb23c None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:51:18.330251 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5b90ec-cf61-44ab-87ac-6ed105afb23c None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:51:18.330557 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5b90ec-cf61-44ab-87ac-6ed105afb23c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:51:18.350594 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-7a5b90ec-cf61-44ab-87ac-6ed105afb23c None None] MFA Rules not processed for user `c7922d0a1a7c4e808d0b2a80dca0da40`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:51:18.351813 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-7a5b90ec-cf61-44ab-87ac-6ed105afb23c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:51:18.401976 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1184/2343] 10.222.0.36 () {58 vars in 1018 bytes} [Wed Jul 27 10:51:18 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:51:18.413615 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b26a324-5474-4be9-b0a8-93a61eca2f65 None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:51:18.415309 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-9b26a324-5474-4be9-b0a8-93a61eca2f65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:51:18.428682 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b26a324-5474-4be9-b0a8-93a61eca2f65 None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:51:18.432286 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9457a6-9a95-4050-bae2-728dfeb1da9e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7922d0a1a7c4e808d0b2a80dca0da40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:51:18.432886 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9457a6-9a95-4050-bae2-728dfeb1da9e service neutron] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:51:18.433168 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9457a6-9a95-4050-bae2-728dfeb1da9e service neutron] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:51:18.433444 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9457a6-9a95-4050-bae2-728dfeb1da9e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:51:18.440326 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9457a6-9a95-4050-bae2-728dfeb1da9e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:51:18.441749 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9457a6-9a95-4050-bae2-728dfeb1da9e service neutron] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:51:18.449480 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1160/2344] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:51:18 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 10:51:18.531947 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a80d2a8-eb45-4b96-8767-50673042764b None None] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:51:18.532237 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a80d2a8-eb45-4b96-8767-50673042764b None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:51:18.532538 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a80d2a8-eb45-4b96-8767-50673042764b None None] PATH_INFO: `/` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:51:18.533922 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1185/2345] 10.222.0.36 () {54 vars in 921 bytes} [Wed Jul 27 10:51:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 10:51:18.538782 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4691d26-7b24-4587-bf79-3269237c6cc8 None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:51:18.539098 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4691d26-7b24-4587-bf79-3269237c6cc8 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:51:18.539375 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4691d26-7b24-4587-bf79-3269237c6cc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:51:18.560066 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-f4691d26-7b24-4587-bf79-3269237c6cc8 None None] MFA Rules not processed for user `e059d8b6c28c4fcba925d05d7c98e4ac`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:51:18.561258 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-f4691d26-7b24-4587-bf79-3269237c6cc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:51:18.570480 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1161/2346] 10.222.0.36 () {58 vars in 1018 bytes} [Wed Jul 27 10:51:18 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:51:18.581055 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a80d2a8-eb45-4b96-8767-50673042764b None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:51:18.582750 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-3a80d2a8-eb45-4b96-8767-50673042764b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:51:18.595925 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a80d2a8-eb45-4b96-8767-50673042764b None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:51:18.599767 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3901ea26-93d8-40d7-b744-8eb8eaba3313 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7922d0a1a7c4e808d0b2a80dca0da40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:51:18.600368 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3901ea26-93d8-40d7-b744-8eb8eaba3313 service neutron] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:51:18.600672 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3901ea26-93d8-40d7-b744-8eb8eaba3313 service neutron] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:51:18.600905 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3901ea26-93d8-40d7-b744-8eb8eaba3313 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:51:18.611944 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3901ea26-93d8-40d7-b744-8eb8eaba3313 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:51:18.613370 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3901ea26-93d8-40d7-b744-8eb8eaba3313 service neutron] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:51:18.624903 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1186/2347] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:51:18 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 10:51:25.551290 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6e1746-4055-472d-9389-ac107f3e5c89 None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:51:25.551750 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6e1746-4055-472d-9389-ac107f3e5c89 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:51:25.551873 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6e1746-4055-472d-9389-ac107f3e5c89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:51:25.576375 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-fc6e1746-4055-472d-9389-ac107f3e5c89 None None] MFA Rules not processed for user `3d9fed24b4ba44b0b5961a603c4362c4`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:51:25.577523 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-fc6e1746-4055-472d-9389-ac107f3e5c89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:51:25.584998 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1162/2348] 10.222.0.36 () {58 vars in 1009 bytes} [Wed Jul 27 10:51:25 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:51:25.598691 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3901ea26-93d8-40d7-b744-8eb8eaba3313 service neutron] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:51:25.613956 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3901ea26-93d8-40d7-b744-8eb8eaba3313 service neutron] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:51:25.619035 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4d5c0b-e658-463f-ac76-dc62df4c21a9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89ac506eefcd4e42a84b44fa15fb3975', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:51:25.619581 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4d5c0b-e658-463f-ac76-dc62df4c21a9 service swift] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:51:25.619865 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4d5c0b-e658-463f-ac76-dc62df4c21a9 service swift] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:51:25.620142 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4d5c0b-e658-463f-ac76-dc62df4c21a9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:51:25.629106 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4d5c0b-e658-463f-ac76-dc62df4c21a9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:51:25.630719 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4d5c0b-e658-463f-ac76-dc62df4c21a9 service swift] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:51:25.639860 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1187/2349] 10.222.0.36 () {60 vars in 1384 bytes} [Wed Jul 27 10:51:25 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 27 10:51:35.566197 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=43 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:51:35.636177 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=61 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:51:44.499530 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa72132-f81c-4f17-b1de-422abfd7d6a2 None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:51:44.500308 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa72132-f81c-4f17-b1de-422abfd7d6a2 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:51:44.500410 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa72132-f81c-4f17-b1de-422abfd7d6a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:51:44.521087 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-afa72132-f81c-4f17-b1de-422abfd7d6a2 None None] MFA Rules not processed for user `5ae9a5696bdc48778835da7a311f20c7`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:51:44.522669 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-afa72132-f81c-4f17-b1de-422abfd7d6a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:51:44.598082 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1163/2350] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:51:44 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:51:45.871201 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a043738-5e4b-45c6-ab34-fbf6e06022c3 None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:51:45.871686 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a043738-5e4b-45c6-ab34-fbf6e06022c3 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:51:45.871835 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a043738-5e4b-45c6-ab34-fbf6e06022c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:51:45.897304 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-0a043738-5e4b-45c6-ab34-fbf6e06022c3 None None] MFA Rules not processed for user `3d9fed24b4ba44b0b5961a603c4362c4`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:51:45.899236 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-0a043738-5e4b-45c6-ab34-fbf6e06022c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:51:45.907061 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1188/2351] 10.222.0.36 () {58 vars in 967 bytes} [Wed Jul 27 10:51:45 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:51:45.920848 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa72132-f81c-4f17-b1de-422abfd7d6a2 None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:51:45.932886 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa72132-f81c-4f17-b1de-422abfd7d6a2 None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:51:45.937912 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5262349f-9d16-419d-abea-8055da728e32 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89ac506eefcd4e42a84b44fa15fb3975', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:51:45.938534 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5262349f-9d16-419d-abea-8055da728e32 service swift] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:51:45.938851 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5262349f-9d16-419d-abea-8055da728e32 service swift] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:51:45.939132 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5262349f-9d16-419d-abea-8055da728e32 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:51:45.947487 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5262349f-9d16-419d-abea-8055da728e32 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:51:45.949066 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5262349f-9d16-419d-abea-8055da728e32 service swift] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:51:45.957212 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1164/2352] 10.222.0.36 () {60 vars in 1384 bytes} [Wed Jul 27 10:51:45 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 27 10:51:55.891713 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=3 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:51:55.953921 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=31 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:52:27.105090 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6e76e7-b912-490d-ba87-a762cb8e4a17 None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:52:27.105617 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6e76e7-b912-490d-ba87-a762cb8e4a17 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:52:27.105728 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6e76e7-b912-490d-ba87-a762cb8e4a17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:52:27.124138 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-5b6e76e7-b912-490d-ba87-a762cb8e4a17 None None] MFA Rules not processed for user `9b094c00959246f5a113f600fb07215d`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:52:27.125580 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-5b6e76e7-b912-490d-ba87-a762cb8e4a17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:52:27.181443 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1189/2353] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:52:27 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:52:37.174460 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=25 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:52:40.832971 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcfb2b3-1559-46a2-93c9-9eb9ebdac371 None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:52:40.833956 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcfb2b3-1559-46a2-93c9-9eb9ebdac371 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:52:40.834871 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcfb2b3-1559-46a2-93c9-9eb9ebdac371 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:52:40.860980 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-1dcfb2b3-1559-46a2-93c9-9eb9ebdac371 None None] MFA Rules not processed for user `3d9fed24b4ba44b0b5961a603c4362c4`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:52:40.863265 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-1dcfb2b3-1559-46a2-93c9-9eb9ebdac371 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:52:40.883032 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1165/2354] 10.222.0.36 () {58 vars in 967 bytes} [Wed Jul 27 10:52:40 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:52:40.901936 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6e76e7-b912-490d-ba87-a762cb8e4a17 None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:52:40.919481 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6e76e7-b912-490d-ba87-a762cb8e4a17 None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:52:40.926544 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2430f6b-ab85-46d1-8b50-e3beb271205f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89ac506eefcd4e42a84b44fa15fb3975', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:52:40.929473 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2430f6b-ab85-46d1-8b50-e3beb271205f service swift] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:52:40.930196 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2430f6b-ab85-46d1-8b50-e3beb271205f service swift] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:52:40.930789 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2430f6b-ab85-46d1-8b50-e3beb271205f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:52:40.949481 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2430f6b-ab85-46d1-8b50-e3beb271205f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:52:40.949481 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2430f6b-ab85-46d1-8b50-e3beb271205f service swift] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:52:40.968580 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1190/2355] 10.222.0.36 () {60 vars in 1384 bytes} [Wed Jul 27 10:52:40 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 27 10:52:42.965008 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f1441a-917e-4087-90d9-458487f8f8b2 None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:52:42.965509 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f1441a-917e-4087-90d9-458487f8f8b2 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:52:42.965615 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f1441a-917e-4087-90d9-458487f8f8b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:52:42.988090 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-31f1441a-917e-4087-90d9-458487f8f8b2 None None] MFA Rules not processed for user `fc1270b519b44320a772d7ad163c873d`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:52:42.989692 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-31f1441a-917e-4087-90d9-458487f8f8b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:52:43.057692 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1166/2356] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:52:42 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:52:46.319092 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7941a71-2587-40af-8ecc-1fed3b95fe9a None None] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:52:46.319787 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7941a71-2587-40af-8ecc-1fed3b95fe9a None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:52:46.319787 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7941a71-2587-40af-8ecc-1fed3b95fe9a None None] PATH_INFO: `/` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:52:46.320993 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1191/2357] 10.222.0.36 () {54 vars in 921 bytes} [Wed Jul 27 10:52:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 10:52:46.355725 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a289ee84-b571-4a3f-aacf-6af57ec9e0d8 None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:52:46.356302 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a289ee84-b571-4a3f-aacf-6af57ec9e0d8 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:52:46.356302 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a289ee84-b571-4a3f-aacf-6af57ec9e0d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:52:46.378032 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-a289ee84-b571-4a3f-aacf-6af57ec9e0d8 None None] MFA Rules not processed for user `16cc6179e9a04b6398fbacb40ad1c555`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:52:46.379389 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-a289ee84-b571-4a3f-aacf-6af57ec9e0d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:52:46.389644 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1167/2358] 10.222.0.36 () {58 vars in 1018 bytes} [Wed Jul 27 10:52:46 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:52:46.404983 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7941a71-2587-40af-8ecc-1fed3b95fe9a None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:52:46.406770 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-b7941a71-2587-40af-8ecc-1fed3b95fe9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:52:46.423789 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7941a71-2587-40af-8ecc-1fed3b95fe9a None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:52:46.428807 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8e97ab-bd21-4891-a5b2-ee47da98f953 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16cc6179e9a04b6398fbacb40ad1c555', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:52:46.429432 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8e97ab-bd21-4891-a5b2-ee47da98f953 service placement] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:52:46.429719 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8e97ab-bd21-4891-a5b2-ee47da98f953 service placement] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:52:46.430005 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8e97ab-bd21-4891-a5b2-ee47da98f953 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:52:46.439153 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8e97ab-bd21-4891-a5b2-ee47da98f953 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:52:46.440727 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8e97ab-bd21-4891-a5b2-ee47da98f953 service placement] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:52:46.451301 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1192/2359] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:52:46 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 10:52:50.596725 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a289ee84-b571-4a3f-aacf-6af57ec9e0d8 None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:52:50.598308 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-a289ee84-b571-4a3f-aacf-6af57ec9e0d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:52:50.673002 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a289ee84-b571-4a3f-aacf-6af57ec9e0d8 None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:52:50.677583 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7d5bed-3664-425d-a261-cc4cbd560050 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:52:50.678224 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7d5bed-3664-425d-a261-cc4cbd560050 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:52:50.678510 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7d5bed-3664-425d-a261-cc4cbd560050 admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:52:50.678820 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7d5bed-3664-425d-a261-cc4cbd560050 admin admin] PATH_INFO: `/v3/users/a90214be44404d88b8814bed43160411` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:52:50.679991 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7d5bed-3664-425d-a261-cc4cbd560050 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a90214be44404d88b8814bed43160411)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:52:50.681919 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7d5bed-3664-425d-a261-cc4cbd560050 admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:52:50.734936 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-4b7d5bed-3664-425d-a261-cc4cbd560050 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a90214be44404d88b8814bed43160411'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:52:50.742403 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-4b7d5bed-3664-425d-a261-cc4cbd560050 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a90214be44404d88b8814bed43160411'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:52:50.751324 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-4b7d5bed-3664-425d-a261-cc4cbd560050 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a90214be44404d88b8814bed43160411'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:52:50.751939 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-4b7d5bed-3664-425d-a261-cc4cbd560050 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a90214be44404d88b8814bed43160411'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:52:50.752674 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-4b7d5bed-3664-425d-a261-cc4cbd560050 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a90214be44404d88b8814bed43160411'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:52:50.771324 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1168/2360] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Jul 27 10:52:50 2022] DELETE /identity/v3/users/a90214be44404d88b8814bed43160411 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:52:50.997293 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8e97ab-bd21-4891-a5b2-ee47da98f953 service placement] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:52:51.008910 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8e97ab-bd21-4891-a5b2-ee47da98f953 service placement] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:52:51.069422 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4ade3f-7a5d-44f1-8e18-549638dd5a48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:52:51.070776 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4ade3f-7a5d-44f1-8e18-549638dd5a48 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:52:51.071068 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4ade3f-7a5d-44f1-8e18-549638dd5a48 admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:52:51.071355 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4ade3f-7a5d-44f1-8e18-549638dd5a48 admin admin] PATH_INFO: `/v3/projects/b961e9565f174504a1f8d4b36dc73f50` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:52:51.072563 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4ade3f-7a5d-44f1-8e18-549638dd5a48 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b961e9565f174504a1f8d4b36dc73f50)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:52:51.074479 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4ade3f-7a5d-44f1-8e18-549638dd5a48 admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:52:51.136297 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-fe4ade3f-7a5d-44f1-8e18-549638dd5a48 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b961e9565f174504a1f8d4b36dc73f50', 'request_id': 'req-fe4ade3f-7a5d-44f1-8e18-549638dd5a48'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:52:51.142741 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-fe4ade3f-7a5d-44f1-8e18-549638dd5a48 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b961e9565f174504a1f8d4b36dc73f50', 'request_id': 'req-fe4ade3f-7a5d-44f1-8e18-549638dd5a48'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:52:51.154576 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-fe4ade3f-7a5d-44f1-8e18-549638dd5a48 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b961e9565f174504a1f8d4b36dc73f50', 'request_id': 'req-fe4ade3f-7a5d-44f1-8e18-549638dd5a48'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:52:51.165873 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-fe4ade3f-7a5d-44f1-8e18-549638dd5a48 admin admin] The token cache is being invalidate because project b961e9565f174504a1f8d4b36dc73f50 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86827) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 10:52:51.166161 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-fe4ade3f-7a5d-44f1-8e18-549638dd5a48 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:52:51.166834 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-fe4ade3f-7a5d-44f1-8e18-549638dd5a48 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:52:51.168606 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1193/2361] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Jul 27 10:52:50 2022] DELETE /identity/v3/projects/b961e9565f174504a1f8d4b36dc73f50 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:52:51.173812 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7d5bed-3664-425d-a261-cc4cbd560050 admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:52:51.175236 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-4b7d5bed-3664-425d-a261-cc4cbd560050 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:52:51.192271 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7d5bed-3664-425d-a261-cc4cbd560050 admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:52:51.200839 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:52:51.262307 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523445f6-2c25-44e2-ac66-c2c3e4220bb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:52:51.263514 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523445f6-2c25-44e2-ac66-c2c3e4220bb1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:52:51.263797 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523445f6-2c25-44e2-ac66-c2c3e4220bb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:52:51.264069 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523445f6-2c25-44e2-ac66-c2c3e4220bb1 admin admin] PATH_INFO: `/v3/users/58f53be7efe94134bd08c64dc7da71f1` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:52:51.265227 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523445f6-2c25-44e2-ac66-c2c3e4220bb1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=58f53be7efe94134bd08c64dc7da71f1)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:52:51.267182 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523445f6-2c25-44e2-ac66-c2c3e4220bb1 admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:52:51.314775 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-523445f6-2c25-44e2-ac66-c2c3e4220bb1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '58f53be7efe94134bd08c64dc7da71f1'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:52:51.321484 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-523445f6-2c25-44e2-ac66-c2c3e4220bb1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '58f53be7efe94134bd08c64dc7da71f1'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:52:51.329292 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-523445f6-2c25-44e2-ac66-c2c3e4220bb1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '58f53be7efe94134bd08c64dc7da71f1'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:52:51.329908 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-523445f6-2c25-44e2-ac66-c2c3e4220bb1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '58f53be7efe94134bd08c64dc7da71f1'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:52:51.330513 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-523445f6-2c25-44e2-ac66-c2c3e4220bb1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '58f53be7efe94134bd08c64dc7da71f1'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:52:51.348006 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1169/2362] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Jul 27 10:52:51 2022] DELETE /identity/v3/users/58f53be7efe94134bd08c64dc7da71f1 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:52:51.515531 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4ade3f-7a5d-44f1-8e18-549638dd5a48 admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:52:51.517000 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-fe4ade3f-7a5d-44f1-8e18-549638dd5a48 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:52:51.529864 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4ade3f-7a5d-44f1-8e18-549638dd5a48 admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:52:51.586195 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e145a1-f0e4-4354-bfaa-d90d569c5636 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:52:51.586863 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e145a1-f0e4-4354-bfaa-d90d569c5636 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:52:51.587149 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e145a1-f0e4-4354-bfaa-d90d569c5636 admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:52:51.587428 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e145a1-f0e4-4354-bfaa-d90d569c5636 admin admin] PATH_INFO: `/v3/projects/d0a98dfcb8d14e208cbd12488b4cf633` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:52:51.588580 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e145a1-f0e4-4354-bfaa-d90d569c5636 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d0a98dfcb8d14e208cbd12488b4cf633)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:52:51.590485 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e145a1-f0e4-4354-bfaa-d90d569c5636 admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:52:51.596840 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:52:51.639231 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-46e145a1-f0e4-4354-bfaa-d90d569c5636 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd0a98dfcb8d14e208cbd12488b4cf633', 'request_id': 'req-46e145a1-f0e4-4354-bfaa-d90d569c5636'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:52:51.645367 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-46e145a1-f0e4-4354-bfaa-d90d569c5636 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd0a98dfcb8d14e208cbd12488b4cf633', 'request_id': 'req-46e145a1-f0e4-4354-bfaa-d90d569c5636'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:52:51.654103 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-46e145a1-f0e4-4354-bfaa-d90d569c5636 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd0a98dfcb8d14e208cbd12488b4cf633', 'request_id': 'req-46e145a1-f0e4-4354-bfaa-d90d569c5636'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:52:51.664608 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-46e145a1-f0e4-4354-bfaa-d90d569c5636 admin admin] The token cache is being invalidate because project d0a98dfcb8d14e208cbd12488b4cf633 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86827) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 10:52:51.664897 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-46e145a1-f0e4-4354-bfaa-d90d569c5636 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:52:51.665521 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-46e145a1-f0e4-4354-bfaa-d90d569c5636 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:52:51.667077 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1194/2363] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Jul 27 10:52:51 2022] DELETE /identity/v3/projects/d0a98dfcb8d14e208cbd12488b4cf633 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:52:52.016232 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523445f6-2c25-44e2-ac66-c2c3e4220bb1 admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:52:52.017676 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-523445f6-2c25-44e2-ac66-c2c3e4220bb1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:52:52.074881 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523445f6-2c25-44e2-ac66-c2c3e4220bb1 admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:52:52.119535 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:52:52.127267 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89097a1a-f039-4e05-a8f3-d916f04aa0f0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7922d0a1a7c4e808d0b2a80dca0da40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:52:52.128295 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89097a1a-f039-4e05-a8f3-d916f04aa0f0 service neutron] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:52:52.128719 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89097a1a-f039-4e05-a8f3-d916f04aa0f0 service neutron] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:52:52.129023 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89097a1a-f039-4e05-a8f3-d916f04aa0f0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:52:52.131231 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-89097a1a-f039-4e05-a8f3-d916f04aa0f0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:52:52.201116 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89097a1a-f039-4e05-a8f3-d916f04aa0f0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:52:52.202456 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89097a1a-f039-4e05-a8f3-d916f04aa0f0 service neutron] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:52:52.213088 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1170/2364] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:52:52 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 10:52:52.281274 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e145a1-f0e4-4354-bfaa-d90d569c5636 admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:52:52.287390 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-46e145a1-f0e4-4354-bfaa-d90d569c5636 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:52:52.299808 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e145a1-f0e4-4354-bfaa-d90d569c5636 admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:52:52.303689 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2aba2cd-aa8e-4c0b-a7e3-eeafed6be4d7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7922d0a1a7c4e808d0b2a80dca0da40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:52:52.304315 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2aba2cd-aa8e-4c0b-a7e3-eeafed6be4d7 service neutron] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:52:52.304593 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2aba2cd-aa8e-4c0b-a7e3-eeafed6be4d7 service neutron] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:52:52.304974 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2aba2cd-aa8e-4c0b-a7e3-eeafed6be4d7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:52:52.306740 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-a2aba2cd-aa8e-4c0b-a7e3-eeafed6be4d7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:52:52.322380 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2aba2cd-aa8e-4c0b-a7e3-eeafed6be4d7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:52:52.323828 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2aba2cd-aa8e-4c0b-a7e3-eeafed6be4d7 service neutron] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:52:52.336059 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1195/2365] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:52:52 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 10:52:54.526745 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89097a1a-f039-4e05-a8f3-d916f04aa0f0 service neutron] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:52:54.528465 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-89097a1a-f039-4e05-a8f3-d916f04aa0f0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:52:54.539354 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89097a1a-f039-4e05-a8f3-d916f04aa0f0 service neutron] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:52:54.543433 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9e9879-d856-409d-b987-a59ba67678ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7922d0a1a7c4e808d0b2a80dca0da40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:52:54.544134 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9e9879-d856-409d-b987-a59ba67678ab service neutron] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:52:54.544501 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9e9879-d856-409d-b987-a59ba67678ab service neutron] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:52:54.544843 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9e9879-d856-409d-b987-a59ba67678ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:52:54.546679 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-2a9e9879-d856-409d-b987-a59ba67678ab service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:52:54.613123 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9e9879-d856-409d-b987-a59ba67678ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:52:54.614582 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9e9879-d856-409d-b987-a59ba67678ab service neutron] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:52:54.625992 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1171/2366] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:52:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 10:52:57.959778 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2aba2cd-aa8e-4c0b-a7e3-eeafed6be4d7 service neutron] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:52:57.961300 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-a2aba2cd-aa8e-4c0b-a7e3-eeafed6be4d7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:52:57.976163 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2aba2cd-aa8e-4c0b-a7e3-eeafed6be4d7 service neutron] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:52:57.981173 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b417ef8-a82d-4664-91f2-ed73cf4ba0bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:52:57.981821 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b417ef8-a82d-4664-91f2-ed73cf4ba0bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:52:57.982109 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b417ef8-a82d-4664-91f2-ed73cf4ba0bd admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:52:57.982388 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b417ef8-a82d-4664-91f2-ed73cf4ba0bd admin admin] PATH_INFO: `/v3/users/5ae9a5696bdc48778835da7a311f20c7` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:52:57.983589 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b417ef8-a82d-4664-91f2-ed73cf4ba0bd admin admin] RBAC: Authorizing `identity:delete_user(user_id=5ae9a5696bdc48778835da7a311f20c7)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:52:57.985599 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b417ef8-a82d-4664-91f2-ed73cf4ba0bd admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:52:58.042207 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-0b417ef8-a82d-4664-91f2-ed73cf4ba0bd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5ae9a5696bdc48778835da7a311f20c7'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:52:58.049074 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-0b417ef8-a82d-4664-91f2-ed73cf4ba0bd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5ae9a5696bdc48778835da7a311f20c7'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:52:58.056473 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-0b417ef8-a82d-4664-91f2-ed73cf4ba0bd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5ae9a5696bdc48778835da7a311f20c7'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:52:58.057152 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-0b417ef8-a82d-4664-91f2-ed73cf4ba0bd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5ae9a5696bdc48778835da7a311f20c7'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:52:58.057844 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-0b417ef8-a82d-4664-91f2-ed73cf4ba0bd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5ae9a5696bdc48778835da7a311f20c7'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:52:58.076207 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1196/2367] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Jul 27 10:52:57 2022] DELETE /identity/v3/users/5ae9a5696bdc48778835da7a311f20c7 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:52:58.247479 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9e9879-d856-409d-b987-a59ba67678ab service neutron] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:52:58.268229 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9e9879-d856-409d-b987-a59ba67678ab service neutron] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:52:58.331050 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3416ee7a-dc47-4da2-8b0b-e17bbfdd4463 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:52:58.331893 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3416ee7a-dc47-4da2-8b0b-e17bbfdd4463 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:52:58.332177 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3416ee7a-dc47-4da2-8b0b-e17bbfdd4463 admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:52:58.332451 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3416ee7a-dc47-4da2-8b0b-e17bbfdd4463 admin admin] PATH_INFO: `/v3/projects/418d6f237f034c45a7e78538b217ad5e` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:52:58.333646 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3416ee7a-dc47-4da2-8b0b-e17bbfdd4463 admin admin] RBAC: Authorizing `identity:delete_project(project_id=418d6f237f034c45a7e78538b217ad5e)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:52:58.335792 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3416ee7a-dc47-4da2-8b0b-e17bbfdd4463 admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:52:58.372815 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:52:58.402028 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-3416ee7a-dc47-4da2-8b0b-e17bbfdd4463 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '418d6f237f034c45a7e78538b217ad5e', 'request_id': 'req-3416ee7a-dc47-4da2-8b0b-e17bbfdd4463'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:52:58.409295 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-3416ee7a-dc47-4da2-8b0b-e17bbfdd4463 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '418d6f237f034c45a7e78538b217ad5e', 'request_id': 'req-3416ee7a-dc47-4da2-8b0b-e17bbfdd4463'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:52:58.419166 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-3416ee7a-dc47-4da2-8b0b-e17bbfdd4463 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '418d6f237f034c45a7e78538b217ad5e', 'request_id': 'req-3416ee7a-dc47-4da2-8b0b-e17bbfdd4463'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:52:58.440996 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-3416ee7a-dc47-4da2-8b0b-e17bbfdd4463 admin admin] The token cache is being invalidate because project 418d6f237f034c45a7e78538b217ad5e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86828) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 10:52:58.441366 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-3416ee7a-dc47-4da2-8b0b-e17bbfdd4463 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:52:58.442400 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-3416ee7a-dc47-4da2-8b0b-e17bbfdd4463 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:52:58.443862 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1172/2368] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Jul 27 10:52:58 2022] DELETE /identity/v3/projects/418d6f237f034c45a7e78538b217ad5e => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:52:58.448603 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9466a77-a57f-442d-b043-69e22a79b4a1 None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:52:58.448895 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9466a77-a57f-442d-b043-69e22a79b4a1 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:52:58.449177 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9466a77-a57f-442d-b043-69e22a79b4a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:52:58.475224 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-b9466a77-a57f-442d-b043-69e22a79b4a1 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:52:58.476451 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-b9466a77-a57f-442d-b043-69e22a79b4a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:52:58.532763 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:52:58.548209 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1197/2369] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:52:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 10:52:58.556110 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f35632a-a49b-47a8-847f-92c8aee817fb None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:52:58.556717 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f35632a-a49b-47a8-847f-92c8aee817fb None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:52:58.557221 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f35632a-a49b-47a8-847f-92c8aee817fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:52:58.583547 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-9f35632a-a49b-47a8-847f-92c8aee817fb None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:52:58.585201 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-9f35632a-a49b-47a8-847f-92c8aee817fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:52:58.593864 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1173/2370] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:52:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:52:58.598367 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9466a77-a57f-442d-b043-69e22a79b4a1 None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:52:58.610979 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9466a77-a57f-442d-b043-69e22a79b4a1 None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:52:58.616062 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4925698-703c-4e86-a237-eb58c4243e98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:52:58.616783 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4925698-703c-4e86-a237-eb58c4243e98 admin admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:52:58.617065 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4925698-703c-4e86-a237-eb58c4243e98 admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:52:58.617337 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4925698-703c-4e86-a237-eb58c4243e98 admin admin] PATH_INFO: `/v3/domains` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:52:58.618065 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4925698-703c-4e86-a237-eb58c4243e98 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86827) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:52:58.618676 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4925698-703c-4e86-a237-eb58c4243e98 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:52:58.620145 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4925698-703c-4e86-a237-eb58c4243e98 admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:52:58.639204 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1198/2371] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Jul 27 10:52:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:52:58.640708 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f35632a-a49b-47a8-847f-92c8aee817fb None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:52:58.651643 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f35632a-a49b-47a8-847f-92c8aee817fb None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:52:58.657534 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca403e74-aef3-4f03-ae66-76e2e6d57a28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:52:58.658470 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca403e74-aef3-4f03-ae66-76e2e6d57a28 admin admin] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:52:58.658857 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca403e74-aef3-4f03-ae66-76e2e6d57a28 admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:52:58.659274 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca403e74-aef3-4f03-ae66-76e2e6d57a28 admin admin] PATH_INFO: `/v3/projects` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:52:58.660013 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca403e74-aef3-4f03-ae66-76e2e6d57a28 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:52:58.661997 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca403e74-aef3-4f03-ae66-76e2e6d57a28 admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:52:58.698824 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1174/2372] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Jul 27 10:52:58 2022] POST /identity/v3/projects => generated 403 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 10:52:58.703477 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4925698-703c-4e86-a237-eb58c4243e98 admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:52:58.715329 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4925698-703c-4e86-a237-eb58c4243e98 admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:52:58.784258 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b271c000-5f5e-4745-8662-24cf50c49054 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:52:58.784984 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b271c000-5f5e-4745-8662-24cf50c49054 admin admin] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:52:58.785345 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b271c000-5f5e-4745-8662-24cf50c49054 admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:52:58.785689 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b271c000-5f5e-4745-8662-24cf50c49054 admin admin] PATH_INFO: `/v3/users` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:52:58.786445 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b271c000-5f5e-4745-8662-24cf50c49054 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:52:58.788474 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b271c000-5f5e-4745-8662-24cf50c49054 admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:52:58.815045 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1199/2373] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Jul 27 10:52:58 2022] POST /identity/v3/users => generated 302 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 10:52:58.818726 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca403e74-aef3-4f03-ae66-76e2e6d57a28 admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:52:58.833097 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca403e74-aef3-4f03-ae66-76e2e6d57a28 admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:52:58.838132 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808b9143-5801-4618-be34-7f117971aad1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:52:58.839413 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808b9143-5801-4618-be34-7f117971aad1 admin admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:52:58.839413 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808b9143-5801-4618-be34-7f117971aad1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:52:58.839413 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808b9143-5801-4618-be34-7f117971aad1 admin admin] PATH_INFO: `/v3/roles` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:52:58.840178 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808b9143-5801-4618-be34-7f117971aad1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:52:58.842464 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808b9143-5801-4618-be34-7f117971aad1 admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:52:58.854418 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1175/2374] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 10:52:58 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 10:52:58.859382 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b271c000-5f5e-4745-8662-24cf50c49054 admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:52:58.873676 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b271c000-5f5e-4745-8662-24cf50c49054 admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:52:58.880159 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109ad83e-d018-473c-ad3f-060943abbfd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:52:58.881156 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109ad83e-d018-473c-ad3f-060943abbfd2 admin admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:52:58.881443 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109ad83e-d018-473c-ad3f-060943abbfd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:52:58.881724 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109ad83e-d018-473c-ad3f-060943abbfd2 admin admin] PATH_INFO: `/v3/roles` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:52:58.882423 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109ad83e-d018-473c-ad3f-060943abbfd2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:52:58.884076 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109ad83e-d018-473c-ad3f-060943abbfd2 admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:52:58.897034 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1200/2375] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 10:52:58 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 10:52:58.901718 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808b9143-5801-4618-be34-7f117971aad1 admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:52:58.912429 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808b9143-5801-4618-be34-7f117971aad1 admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:52:58.917488 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88076b35-bfdd-406b-a0db-5f2b75e65c45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:52:58.918163 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88076b35-bfdd-406b-a0db-5f2b75e65c45 admin admin] REQUEST_METHOD: `PUT` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:52:58.918989 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88076b35-bfdd-406b-a0db-5f2b75e65c45 admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:52:58.919270 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88076b35-bfdd-406b-a0db-5f2b75e65c45 admin admin] PATH_INFO: `/v3/projects/9898a080cae9462d882894543dfec862/users/f60775b75b1e47e4b2b1f27f3f5f3a3f/roles/4c9fc0b3baed441da2126be41cc6b04a` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:52:58.934215 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88076b35-bfdd-406b-a0db-5f2b75e65c45 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9898a080cae9462d882894543dfec862, user_id=f60775b75b1e47e4b2b1f27f3f5f3a3f, role_id=4c9fc0b3baed441da2126be41cc6b04a)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:52:58.938329 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88076b35-bfdd-406b-a0db-5f2b75e65c45 admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:52:58.958846 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1176/2376] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 10:52:58 2022] PUT /identity/v3/projects/9898a080cae9462d882894543dfec862/users/f60775b75b1e47e4b2b1f27f3f5f3a3f/roles/4c9fc0b3baed441da2126be41cc6b04a => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:52:58.969802 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e93ed1b-f7c1-4c43-b953-047cc0ce22c8 None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:52:58.970698 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e93ed1b-f7c1-4c43-b953-047cc0ce22c8 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:52:58.970987 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e93ed1b-f7c1-4c43-b953-047cc0ce22c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:52:59.001077 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-7e93ed1b-f7c1-4c43-b953-047cc0ce22c8 None None] MFA Rules not processed for user `f60775b75b1e47e4b2b1f27f3f5f3a3f`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:52:59.002505 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-7e93ed1b-f7c1-4c43-b953-047cc0ce22c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:52:59.068366 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1201/2377] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:52:58 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:52:59.072769 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88076b35-bfdd-406b-a0db-5f2b75e65c45 admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:52:59.137259 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88076b35-bfdd-406b-a0db-5f2b75e65c45 admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:52:59.142051 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae252c1-4180-4cc5-9291-6a136eff3d1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:52:59.142702 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae252c1-4180-4cc5-9291-6a136eff3d1f admin admin] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:52:59.142997 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae252c1-4180-4cc5-9291-6a136eff3d1f admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:52:59.143273 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae252c1-4180-4cc5-9291-6a136eff3d1f admin admin] PATH_INFO: `/v3/projects` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:52:59.143954 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae252c1-4180-4cc5-9291-6a136eff3d1f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:52:59.147017 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae252c1-4180-4cc5-9291-6a136eff3d1f admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:52:59.175846 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1177/2378] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Jul 27 10:52:59 2022] POST /identity/v3/projects => generated 403 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 10:52:59.179940 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e93ed1b-f7c1-4c43-b953-047cc0ce22c8 None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:52:59.191836 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e93ed1b-f7c1-4c43-b953-047cc0ce22c8 None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:52:59.237837 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:52:59.252648 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787f5842-e9ff-4910-a226-e3ab5e28daea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:52:59.253279 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787f5842-e9ff-4910-a226-e3ab5e28daea admin admin] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:52:59.253561 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787f5842-e9ff-4910-a226-e3ab5e28daea admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:52:59.253838 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787f5842-e9ff-4910-a226-e3ab5e28daea admin admin] PATH_INFO: `/v3/users` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:52:59.254541 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787f5842-e9ff-4910-a226-e3ab5e28daea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:52:59.256538 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787f5842-e9ff-4910-a226-e3ab5e28daea admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:52:59.279716 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1202/2379] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Jul 27 10:52:59 2022] POST /identity/v3/users => generated 308 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 10:52:59.283706 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae252c1-4180-4cc5-9291-6a136eff3d1f admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:52:59.294460 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae252c1-4180-4cc5-9291-6a136eff3d1f admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:52:59.299654 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65326e9-80e7-419b-8fab-f227d8abd5a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:52:59.300303 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65326e9-80e7-419b-8fab-f227d8abd5a0 admin admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:52:59.300587 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65326e9-80e7-419b-8fab-f227d8abd5a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:52:59.300861 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65326e9-80e7-419b-8fab-f227d8abd5a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:52:59.301560 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65326e9-80e7-419b-8fab-f227d8abd5a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:52:59.303306 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65326e9-80e7-419b-8fab-f227d8abd5a0 admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:52:59.315437 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1178/2380] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 10:52:59 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 10:52:59.320316 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787f5842-e9ff-4910-a226-e3ab5e28daea admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:52:59.330971 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787f5842-e9ff-4910-a226-e3ab5e28daea admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:52:59.335655 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326be0c9-d5c5-41fe-931a-348bf6a2e5eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:52:59.336341 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326be0c9-d5c5-41fe-931a-348bf6a2e5eb admin admin] REQUEST_METHOD: `PUT` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:52:59.336619 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326be0c9-d5c5-41fe-931a-348bf6a2e5eb admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:52:59.336896 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326be0c9-d5c5-41fe-931a-348bf6a2e5eb admin admin] PATH_INFO: `/v3/projects/d64f58c13548421686445d22bb2288aa/users/dcf45602ac7946e2bfb5c4ebb3d3c2bc/roles/21156560eb6b436bb567a40b4976f3f4` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:52:59.355036 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326be0c9-d5c5-41fe-931a-348bf6a2e5eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=d64f58c13548421686445d22bb2288aa, user_id=dcf45602ac7946e2bfb5c4ebb3d3c2bc, role_id=21156560eb6b436bb567a40b4976f3f4)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:52:59.358496 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326be0c9-d5c5-41fe-931a-348bf6a2e5eb admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:52:59.372811 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1203/2381] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 10:52:59 2022] PUT /identity/v3/projects/d64f58c13548421686445d22bb2288aa/users/dcf45602ac7946e2bfb5c4ebb3d3c2bc/roles/21156560eb6b436bb567a40b4976f3f4 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:52:59.376981 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65326e9-80e7-419b-8fab-f227d8abd5a0 admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:52:59.386898 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65326e9-80e7-419b-8fab-f227d8abd5a0 admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:52:59.406711 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=90,keystone:DELETE=6,keystone:INSERT=4 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:52:59.449679 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3b8e0f-0cbb-4db7-9e4f-398d1fa1b1b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:52:59.450382 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3b8e0f-0cbb-4db7-9e4f-398d1fa1b1b4 admin admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:52:59.450711 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3b8e0f-0cbb-4db7-9e4f-398d1fa1b1b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:52:59.450987 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3b8e0f-0cbb-4db7-9e4f-398d1fa1b1b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:52:59.451690 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3b8e0f-0cbb-4db7-9e4f-398d1fa1b1b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:52:59.453300 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3b8e0f-0cbb-4db7-9e4f-398d1fa1b1b4 admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:52:59.467471 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1179/2382] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 10:52:59 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 10:52:59.468619 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326be0c9-d5c5-41fe-931a-348bf6a2e5eb admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:52:59.479100 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326be0c9-d5c5-41fe-931a-348bf6a2e5eb admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:52:59.486589 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad27b92-865b-4ecf-be60-0762f6526df8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:52:59.487339 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad27b92-865b-4ecf-be60-0762f6526df8 admin admin] REQUEST_METHOD: `PUT` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:52:59.487647 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad27b92-865b-4ecf-be60-0762f6526df8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:52:59.487947 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad27b92-865b-4ecf-be60-0762f6526df8 admin admin] PATH_INFO: `/v3/projects/d64f58c13548421686445d22bb2288aa/users/dcf45602ac7946e2bfb5c4ebb3d3c2bc/roles/21156560eb6b436bb567a40b4976f3f4` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:52:59.490122 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad27b92-865b-4ecf-be60-0762f6526df8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d64f58c13548421686445d22bb2288aa, user_id=dcf45602ac7946e2bfb5c4ebb3d3c2bc, role_id=21156560eb6b436bb567a40b4976f3f4)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:52:59.492785 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad27b92-865b-4ecf-be60-0762f6526df8 admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:52:59.504554 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1204/2383] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 10:52:59 2022] PUT /identity/v3/projects/d64f58c13548421686445d22bb2288aa/users/dcf45602ac7946e2bfb5c4ebb3d3c2bc/roles/21156560eb6b436bb567a40b4976f3f4 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:52:59.516705 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c5b8ea-e0e1-438e-8d71-1d1f27d335b0 None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:52:59.517113 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c5b8ea-e0e1-438e-8d71-1d1f27d335b0 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:52:59.517401 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c5b8ea-e0e1-438e-8d71-1d1f27d335b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:52:59.545720 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-89c5b8ea-e0e1-438e-8d71-1d1f27d335b0 None None] MFA Rules not processed for user `dcf45602ac7946e2bfb5c4ebb3d3c2bc`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:52:59.546959 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-89c5b8ea-e0e1-438e-8d71-1d1f27d335b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:52:59.616966 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1180/2384] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:52:59 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:52:59.628773 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad27b92-865b-4ecf-be60-0762f6526df8 admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:52:59.630449 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-7ad27b92-865b-4ecf-be60-0762f6526df8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:52:59.687763 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad27b92-865b-4ecf-be60-0762f6526df8 admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:52:59.691998 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039bd7c7-2671-4c48-9c44-a83e0264fafa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '864d35f268cf45148756109759bb2743', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:52:59.692623 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039bd7c7-2671-4c48-9c44-a83e0264fafa service cinder] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:52:59.692908 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039bd7c7-2671-4c48-9c44-a83e0264fafa service cinder] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:52:59.693185 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039bd7c7-2671-4c48-9c44-a83e0264fafa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:52:59.704543 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039bd7c7-2671-4c48-9c44-a83e0264fafa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:52:59.705972 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039bd7c7-2671-4c48-9c44-a83e0264fafa service cinder] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:52:59.716078 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1205/2385] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:52:59 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 10:52:59.850164 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c5b8ea-e0e1-438e-8d71-1d1f27d335b0 None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:52:59.915761 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c5b8ea-e0e1-438e-8d71-1d1f27d335b0 None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:52:59.920526 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd55d44-eb9b-4450-aee2-8f3a5c31c42a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:52:59.921167 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd55d44-eb9b-4450-aee2-8f3a5c31c42a admin admin] REQUEST_METHOD: `DELETE` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:52:59.921455 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd55d44-eb9b-4450-aee2-8f3a5c31c42a admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:52:59.921738 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd55d44-eb9b-4450-aee2-8f3a5c31c42a admin admin] PATH_INFO: `/v3/users/f60775b75b1e47e4b2b1f27f3f5f3a3f` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:52:59.922944 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd55d44-eb9b-4450-aee2-8f3a5c31c42a admin admin] RBAC: Authorizing `identity:delete_user(user_id=f60775b75b1e47e4b2b1f27f3f5f3a3f)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:52:59.924871 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd55d44-eb9b-4450-aee2-8f3a5c31c42a admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:52:59.970375 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:52:59.972365 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-ebd55d44-eb9b-4450-aee2-8f3a5c31c42a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f60775b75b1e47e4b2b1f27f3f5f3a3f'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:52:59.980695 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-ebd55d44-eb9b-4450-aee2-8f3a5c31c42a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f60775b75b1e47e4b2b1f27f3f5f3a3f'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:52:59.988155 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-ebd55d44-eb9b-4450-aee2-8f3a5c31c42a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f60775b75b1e47e4b2b1f27f3f5f3a3f'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:52:59.988916 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-ebd55d44-eb9b-4450-aee2-8f3a5c31c42a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f60775b75b1e47e4b2b1f27f3f5f3a3f'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:52:59.989676 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-ebd55d44-eb9b-4450-aee2-8f3a5c31c42a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f60775b75b1e47e4b2b1f27f3f5f3a3f'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:53:00.008040 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1181/2386] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Jul 27 10:52:59 2022] DELETE /identity/v3/users/f60775b75b1e47e4b2b1f27f3f5f3a3f => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:53:00.013660 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db42286-3ffc-434d-a69e-0342dc985f52 None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:53:00.014166 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db42286-3ffc-434d-a69e-0342dc985f52 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:53:00.014263 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db42286-3ffc-434d-a69e-0342dc985f52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:53:00.033044 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-0db42286-3ffc-434d-a69e-0342dc985f52 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:53:00.034245 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-0db42286-3ffc-434d-a69e-0342dc985f52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:53:00.093343 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1206/2387] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:53:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:53:00.103076 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd55d44-eb9b-4450-aee2-8f3a5c31c42a admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:53:00.104668 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-ebd55d44-eb9b-4450-aee2-8f3a5c31c42a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:53:00.164628 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd55d44-eb9b-4450-aee2-8f3a5c31c42a admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:53:00.168380 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51eb44a9-7db9-45e6-a0c4-5981ab903882 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7922d0a1a7c4e808d0b2a80dca0da40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:53:00.169441 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51eb44a9-7db9-45e6-a0c4-5981ab903882 service neutron] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:53:00.169726 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51eb44a9-7db9-45e6-a0c4-5981ab903882 service neutron] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:53:00.170003 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51eb44a9-7db9-45e6-a0c4-5981ab903882 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:53:00.178919 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51eb44a9-7db9-45e6-a0c4-5981ab903882 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:53:00.180350 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51eb44a9-7db9-45e6-a0c4-5981ab903882 service neutron] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:53:00.190259 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1182/2388] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:53:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 10:53:00.466343 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db42286-3ffc-434d-a69e-0342dc985f52 None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:53:00.467936 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-0db42286-3ffc-434d-a69e-0342dc985f52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:53:00.480791 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db42286-3ffc-434d-a69e-0342dc985f52 None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:53:00.485581 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1863ff59-6f4e-4701-ad55-a4c54cee29ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:53:00.486231 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1863ff59-6f4e-4701-ad55-a4c54cee29ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:53:00.486518 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1863ff59-6f4e-4701-ad55-a4c54cee29ff admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:53:00.486872 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1863ff59-6f4e-4701-ad55-a4c54cee29ff admin admin] PATH_INFO: `/v3/projects/9898a080cae9462d882894543dfec862` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:53:00.488085 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1863ff59-6f4e-4701-ad55-a4c54cee29ff admin admin] RBAC: Authorizing `identity:delete_project(project_id=9898a080cae9462d882894543dfec862)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:53:00.489994 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1863ff59-6f4e-4701-ad55-a4c54cee29ff admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:53:00.531596 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:53:00.539670 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-1863ff59-6f4e-4701-ad55-a4c54cee29ff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9898a080cae9462d882894543dfec862', 'request_id': 'req-1863ff59-6f4e-4701-ad55-a4c54cee29ff'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:53:00.546205 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-1863ff59-6f4e-4701-ad55-a4c54cee29ff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9898a080cae9462d882894543dfec862', 'request_id': 'req-1863ff59-6f4e-4701-ad55-a4c54cee29ff'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:53:00.553383 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-1863ff59-6f4e-4701-ad55-a4c54cee29ff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9898a080cae9462d882894543dfec862', 'request_id': 'req-1863ff59-6f4e-4701-ad55-a4c54cee29ff'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:53:00.564118 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-1863ff59-6f4e-4701-ad55-a4c54cee29ff admin admin] The token cache is being invalidate because project 9898a080cae9462d882894543dfec862 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86827) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 10:53:00.564402 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-1863ff59-6f4e-4701-ad55-a4c54cee29ff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:53:00.565015 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-1863ff59-6f4e-4701-ad55-a4c54cee29ff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:53:00.566510 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1207/2389] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Jul 27 10:53:00 2022] DELETE /identity/v3/projects/9898a080cae9462d882894543dfec862 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:53:00.570457 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51eb44a9-7db9-45e6-a0c4-5981ab903882 service neutron] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:53:00.580683 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51eb44a9-7db9-45e6-a0c4-5981ab903882 service neutron] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:53:00.636929 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf54348d-7f48-445b-855b-e464700eb33c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:53:00.638181 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf54348d-7f48-445b-855b-e464700eb33c admin admin] REQUEST_METHOD: `DELETE` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:53:00.638465 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf54348d-7f48-445b-855b-e464700eb33c admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:53:00.638785 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf54348d-7f48-445b-855b-e464700eb33c admin admin] PATH_INFO: `/v3/users/dcf45602ac7946e2bfb5c4ebb3d3c2bc` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:53:00.639964 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf54348d-7f48-445b-855b-e464700eb33c admin admin] RBAC: Authorizing `identity:delete_user(user_id=dcf45602ac7946e2bfb5c4ebb3d3c2bc)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:53:00.641889 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf54348d-7f48-445b-855b-e464700eb33c admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:53:00.686236 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-bf54348d-7f48-445b-855b-e464700eb33c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dcf45602ac7946e2bfb5c4ebb3d3c2bc'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:53:00.693037 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-bf54348d-7f48-445b-855b-e464700eb33c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dcf45602ac7946e2bfb5c4ebb3d3c2bc'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:53:00.701095 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-bf54348d-7f48-445b-855b-e464700eb33c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dcf45602ac7946e2bfb5c4ebb3d3c2bc'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:53:00.701794 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-bf54348d-7f48-445b-855b-e464700eb33c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dcf45602ac7946e2bfb5c4ebb3d3c2bc'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:53:00.702483 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-bf54348d-7f48-445b-855b-e464700eb33c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dcf45602ac7946e2bfb5c4ebb3d3c2bc'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:53:00.707104 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:53:00.722706 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1183/2390] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Jul 27 10:53:00 2022] DELETE /identity/v3/users/dcf45602ac7946e2bfb5c4ebb3d3c2bc => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:53:00.993017 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1863ff59-6f4e-4701-ad55-a4c54cee29ff admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:53:00.994748 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-1863ff59-6f4e-4701-ad55-a4c54cee29ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:53:01.009749 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1863ff59-6f4e-4701-ad55-a4c54cee29ff admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:53:01.068014 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b44eb0-2c80-4669-accf-fffd75649811 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:53:01.068681 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b44eb0-2c80-4669-accf-fffd75649811 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:53:01.068928 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b44eb0-2c80-4669-accf-fffd75649811 admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:53:01.069205 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b44eb0-2c80-4669-accf-fffd75649811 admin admin] PATH_INFO: `/v3/projects/d64f58c13548421686445d22bb2288aa` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:53:01.070347 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b44eb0-2c80-4669-accf-fffd75649811 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d64f58c13548421686445d22bb2288aa)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:53:01.072289 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b44eb0-2c80-4669-accf-fffd75649811 admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:53:01.119072 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-f9b44eb0-2c80-4669-accf-fffd75649811 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd64f58c13548421686445d22bb2288aa', 'request_id': 'req-f9b44eb0-2c80-4669-accf-fffd75649811'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:53:01.126153 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-f9b44eb0-2c80-4669-accf-fffd75649811 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd64f58c13548421686445d22bb2288aa', 'request_id': 'req-f9b44eb0-2c80-4669-accf-fffd75649811'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:53:01.133367 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-f9b44eb0-2c80-4669-accf-fffd75649811 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd64f58c13548421686445d22bb2288aa', 'request_id': 'req-f9b44eb0-2c80-4669-accf-fffd75649811'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:53:01.143513 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-f9b44eb0-2c80-4669-accf-fffd75649811 admin admin] The token cache is being invalidate because project d64f58c13548421686445d22bb2288aa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86827) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 10:53:01.143799 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-f9b44eb0-2c80-4669-accf-fffd75649811 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:53:01.144440 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-f9b44eb0-2c80-4669-accf-fffd75649811 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:53:01.145948 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1208/2391] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Jul 27 10:53:00 2022] DELETE /identity/v3/projects/d64f58c13548421686445d22bb2288aa => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:53:01.150913 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52acd7a8-0546-4f05-a2b1-742786420f19 None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:53:01.151202 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52acd7a8-0546-4f05-a2b1-742786420f19 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:53:01.151480 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52acd7a8-0546-4f05-a2b1-742786420f19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:53:01.172169 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-52acd7a8-0546-4f05-a2b1-742786420f19 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:53:01.173371 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-52acd7a8-0546-4f05-a2b1-742786420f19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:53:01.231263 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1184/2392] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:53:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:53:01.239626 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc87257-4f11-49c0-aac7-a62f2fa53c4c None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:53:01.239869 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc87257-4f11-49c0-aac7-a62f2fa53c4c None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:53:01.240125 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc87257-4f11-49c0-aac7-a62f2fa53c4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:53:01.259380 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-acc87257-4f11-49c0-aac7-a62f2fa53c4c None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:53:01.260577 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-acc87257-4f11-49c0-aac7-a62f2fa53c4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:53:01.268245 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1209/2393] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:53:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:53:01.272582 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52acd7a8-0546-4f05-a2b1-742786420f19 None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:53:01.282836 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52acd7a8-0546-4f05-a2b1-742786420f19 None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:53:01.287461 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0cd18c-d4bc-410c-8866-c3a815e3b6ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:53:01.288076 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0cd18c-d4bc-410c-8866-c3a815e3b6ca admin admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:53:01.288368 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0cd18c-d4bc-410c-8866-c3a815e3b6ca admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:53:01.288645 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0cd18c-d4bc-410c-8866-c3a815e3b6ca admin admin] PATH_INFO: `/v3/domains` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:53:01.289347 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0cd18c-d4bc-410c-8866-c3a815e3b6ca admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86828) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:53:01.289655 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0cd18c-d4bc-410c-8866-c3a815e3b6ca admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:53:01.291307 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0cd18c-d4bc-410c-8866-c3a815e3b6ca admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:53:01.303547 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1185/2394] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Jul 27 10:53:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:53:01.307822 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc87257-4f11-49c0-aac7-a62f2fa53c4c None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:53:01.316964 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc87257-4f11-49c0-aac7-a62f2fa53c4c None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:53:01.321528 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172992ab-6c8a-4ea3-a13e-16e8362f68ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:53:01.322124 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172992ab-6c8a-4ea3-a13e-16e8362f68ca admin admin] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:53:01.322402 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172992ab-6c8a-4ea3-a13e-16e8362f68ca admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:53:01.322703 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172992ab-6c8a-4ea3-a13e-16e8362f68ca admin admin] PATH_INFO: `/v3/projects` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:53:01.323368 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172992ab-6c8a-4ea3-a13e-16e8362f68ca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:53:01.325248 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172992ab-6c8a-4ea3-a13e-16e8362f68ca admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:53:01.351554 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1210/2395] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Jul 27 10:53:01 2022] POST /identity/v3/projects => generated 381 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 10:53:01.355682 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0cd18c-d4bc-410c-8866-c3a815e3b6ca admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:53:01.366155 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0cd18c-d4bc-410c-8866-c3a815e3b6ca admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:53:01.421511 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284dba65-190b-45bd-b47c-39db65b69fee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:53:01.422372 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284dba65-190b-45bd-b47c-39db65b69fee admin admin] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:53:01.422737 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284dba65-190b-45bd-b47c-39db65b69fee admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:53:01.423162 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284dba65-190b-45bd-b47c-39db65b69fee admin admin] PATH_INFO: `/v3/users` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:53:01.423936 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284dba65-190b-45bd-b47c-39db65b69fee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:53:01.425855 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284dba65-190b-45bd-b47c-39db65b69fee admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:53:01.446185 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1186/2396] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Jul 27 10:53:01 2022] POST /identity/v3/users => generated 291 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 10:53:01.450410 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172992ab-6c8a-4ea3-a13e-16e8362f68ca admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:53:01.460764 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172992ab-6c8a-4ea3-a13e-16e8362f68ca admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:53:01.465907 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650e886b-4002-49fd-bff9-5972c9c49566 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:53:01.466513 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650e886b-4002-49fd-bff9-5972c9c49566 admin admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:53:01.466831 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650e886b-4002-49fd-bff9-5972c9c49566 admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:53:01.467112 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650e886b-4002-49fd-bff9-5972c9c49566 admin admin] PATH_INFO: `/v3/roles` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:53:01.467808 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650e886b-4002-49fd-bff9-5972c9c49566 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:53:01.469400 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650e886b-4002-49fd-bff9-5972c9c49566 admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:53:01.480225 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1211/2397] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 10:53:01 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 10:53:01.485197 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284dba65-190b-45bd-b47c-39db65b69fee admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:53:01.495598 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284dba65-190b-45bd-b47c-39db65b69fee admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:53:01.501036 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da69bfa3-fe5f-42a1-8300-0d975058787b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:53:01.501657 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da69bfa3-fe5f-42a1-8300-0d975058787b admin admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:53:01.501943 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da69bfa3-fe5f-42a1-8300-0d975058787b admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:53:01.502224 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da69bfa3-fe5f-42a1-8300-0d975058787b admin admin] PATH_INFO: `/v3/roles` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:53:01.502953 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da69bfa3-fe5f-42a1-8300-0d975058787b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:53:01.504639 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da69bfa3-fe5f-42a1-8300-0d975058787b admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:53:01.516227 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1187/2398] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 10:53:01 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 10:53:01.520829 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650e886b-4002-49fd-bff9-5972c9c49566 admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:53:01.531185 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650e886b-4002-49fd-bff9-5972c9c49566 admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:53:01.535719 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c87dd3-0be0-4f8d-9fb1-476aba8ccb2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:53:01.536371 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c87dd3-0be0-4f8d-9fb1-476aba8ccb2b admin admin] REQUEST_METHOD: `PUT` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:53:01.536653 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c87dd3-0be0-4f8d-9fb1-476aba8ccb2b admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:53:01.536928 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c87dd3-0be0-4f8d-9fb1-476aba8ccb2b admin admin] PATH_INFO: `/v3/projects/e1ec6223d4b5462ab93af12fa467c2ff/users/607eee6796384acda899c7c82f55ed21/roles/4c9fc0b3baed441da2126be41cc6b04a` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:53:01.551069 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c87dd3-0be0-4f8d-9fb1-476aba8ccb2b admin admin] RBAC: Authorizing `identity:create_grant(project_id=e1ec6223d4b5462ab93af12fa467c2ff, user_id=607eee6796384acda899c7c82f55ed21, role_id=4c9fc0b3baed441da2126be41cc6b04a)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:53:01.553696 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c87dd3-0be0-4f8d-9fb1-476aba8ccb2b admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:53:01.555846 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:53:01.568682 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1212/2399] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 10:53:01 2022] PUT /identity/v3/projects/e1ec6223d4b5462ab93af12fa467c2ff/users/607eee6796384acda899c7c82f55ed21/roles/4c9fc0b3baed441da2126be41cc6b04a => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:53:01.580703 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcffbbf-a6ab-4c75-a3a5-397483ee9ead None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:53:01.581170 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcffbbf-a6ab-4c75-a3a5-397483ee9ead None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:53:01.581267 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcffbbf-a6ab-4c75-a3a5-397483ee9ead None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:53:01.608692 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-3bcffbbf-a6ab-4c75-a3a5-397483ee9ead None None] MFA Rules not processed for user `607eee6796384acda899c7c82f55ed21`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:53:01.609896 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-3bcffbbf-a6ab-4c75-a3a5-397483ee9ead None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:53:01.649541 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:DELETE=2,keystone:SELECT=95,keystone:INSERT=3 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:53:01.676212 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1188/2400] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:53:01 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:53:01.680755 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c87dd3-0be0-4f8d-9fb1-476aba8ccb2b admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:53:01.742144 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c87dd3-0be0-4f8d-9fb1-476aba8ccb2b admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:53:01.747420 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815d940b-566b-4fd2-a814-ecc619738134 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:53:01.747979 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815d940b-566b-4fd2-a814-ecc619738134 admin admin] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:53:01.748268 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815d940b-566b-4fd2-a814-ecc619738134 admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:53:01.748541 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815d940b-566b-4fd2-a814-ecc619738134 admin admin] PATH_INFO: `/v3/projects` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:53:01.749448 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815d940b-566b-4fd2-a814-ecc619738134 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:53:01.751461 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815d940b-566b-4fd2-a814-ecc619738134 admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:53:01.777181 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1213/2401] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Jul 27 10:53:01 2022] POST /identity/v3/projects => generated 379 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 10:53:01.781302 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcffbbf-a6ab-4c75-a3a5-397483ee9ead None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:53:01.792038 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcffbbf-a6ab-4c75-a3a5-397483ee9ead None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:53:01.849671 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785a324e-1fce-44b0-a73d-7b3c850d680a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:53:01.850297 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785a324e-1fce-44b0-a73d-7b3c850d680a admin admin] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:53:01.850582 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785a324e-1fce-44b0-a73d-7b3c850d680a admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:53:01.850890 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785a324e-1fce-44b0-a73d-7b3c850d680a admin admin] PATH_INFO: `/v3/users` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:53:01.851597 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785a324e-1fce-44b0-a73d-7b3c850d680a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:53:01.853517 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785a324e-1fce-44b0-a73d-7b3c850d680a admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:53:01.874670 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1189/2402] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Jul 27 10:53:01 2022] POST /identity/v3/users => generated 296 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 10:53:01.878988 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815d940b-566b-4fd2-a814-ecc619738134 admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:53:01.890294 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815d940b-566b-4fd2-a814-ecc619738134 admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:53:01.895041 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55e923e-795b-479d-8647-6317e6201c73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:53:01.895662 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55e923e-795b-479d-8647-6317e6201c73 admin admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:53:01.895946 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55e923e-795b-479d-8647-6317e6201c73 admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:53:01.896222 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55e923e-795b-479d-8647-6317e6201c73 admin admin] PATH_INFO: `/v3/roles` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:53:01.896978 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55e923e-795b-479d-8647-6317e6201c73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:53:01.898651 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55e923e-795b-479d-8647-6317e6201c73 admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:53:01.911222 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1214/2403] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 10:53:01 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 10:53:01.915531 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785a324e-1fce-44b0-a73d-7b3c850d680a admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:53:01.925715 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785a324e-1fce-44b0-a73d-7b3c850d680a admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:53:01.930354 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd72d71-df3b-4152-9e5e-7e2704c36036 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:53:01.931053 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd72d71-df3b-4152-9e5e-7e2704c36036 admin admin] REQUEST_METHOD: `PUT` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:53:01.931341 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd72d71-df3b-4152-9e5e-7e2704c36036 admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:53:01.931621 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd72d71-df3b-4152-9e5e-7e2704c36036 admin admin] PATH_INFO: `/v3/projects/61846a14d822478882715e20cac82fa6/users/784a47f327664c38986dd1cf2c238bbe/roles/21156560eb6b436bb567a40b4976f3f4` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:53:01.946564 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd72d71-df3b-4152-9e5e-7e2704c36036 admin admin] RBAC: Authorizing `identity:create_grant(project_id=61846a14d822478882715e20cac82fa6, user_id=784a47f327664c38986dd1cf2c238bbe, role_id=21156560eb6b436bb567a40b4976f3f4)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:53:01.949342 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd72d71-df3b-4152-9e5e-7e2704c36036 admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:53:01.963094 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1190/2404] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 10:53:01 2022] PUT /identity/v3/projects/61846a14d822478882715e20cac82fa6/users/784a47f327664c38986dd1cf2c238bbe/roles/21156560eb6b436bb567a40b4976f3f4 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:53:01.967512 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55e923e-795b-479d-8647-6317e6201c73 admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:53:01.977627 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55e923e-795b-479d-8647-6317e6201c73 admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:53:02.033743 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db40b13d-898e-4e41-948d-fef4ee361d28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:53:02.034289 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db40b13d-898e-4e41-948d-fef4ee361d28 admin admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:53:02.034573 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db40b13d-898e-4e41-948d-fef4ee361d28 admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:53:02.034883 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db40b13d-898e-4e41-948d-fef4ee361d28 admin admin] PATH_INFO: `/v3/roles` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:53:02.035576 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db40b13d-898e-4e41-948d-fef4ee361d28 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:53:02.037191 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db40b13d-898e-4e41-948d-fef4ee361d28 admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:53:02.047102 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1215/2405] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 10:53:01 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 10:53:02.051286 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd72d71-df3b-4152-9e5e-7e2704c36036 admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:53:02.061997 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd72d71-df3b-4152-9e5e-7e2704c36036 admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:53:02.066659 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb15dfbf-706f-401b-80a9-52e01bdba883 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:53:02.067327 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb15dfbf-706f-401b-80a9-52e01bdba883 admin admin] REQUEST_METHOD: `PUT` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:53:02.067612 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb15dfbf-706f-401b-80a9-52e01bdba883 admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:53:02.067891 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb15dfbf-706f-401b-80a9-52e01bdba883 admin admin] PATH_INFO: `/v3/projects/61846a14d822478882715e20cac82fa6/users/784a47f327664c38986dd1cf2c238bbe/roles/21156560eb6b436bb567a40b4976f3f4` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:53:02.069836 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb15dfbf-706f-401b-80a9-52e01bdba883 admin admin] RBAC: Authorizing `identity:create_grant(project_id=61846a14d822478882715e20cac82fa6, user_id=784a47f327664c38986dd1cf2c238bbe, role_id=21156560eb6b436bb567a40b4976f3f4)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:53:02.072517 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb15dfbf-706f-401b-80a9-52e01bdba883 admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:53:02.083216 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1191/2406] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 10:53:02 2022] PUT /identity/v3/projects/61846a14d822478882715e20cac82fa6/users/784a47f327664c38986dd1cf2c238bbe/roles/21156560eb6b436bb567a40b4976f3f4 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 10:53:02.093220 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e467b320-35c0-4a93-a165-2d5f20d9252b None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:53:02.093700 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e467b320-35c0-4a93-a165-2d5f20d9252b None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:53:02.093803 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e467b320-35c0-4a93-a165-2d5f20d9252b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:53:02.121336 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-e467b320-35c0-4a93-a165-2d5f20d9252b None None] MFA Rules not processed for user `784a47f327664c38986dd1cf2c238bbe`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:53:02.123067 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-e467b320-35c0-4a93-a165-2d5f20d9252b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:53:02.172029 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:53:02.191103 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1216/2407] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:53:02 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:53:02.202955 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb15dfbf-706f-401b-80a9-52e01bdba883 admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:53:02.204776 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-eb15dfbf-706f-401b-80a9-52e01bdba883 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:53:02.261249 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb15dfbf-706f-401b-80a9-52e01bdba883 admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:53:02.265519 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705f3251-7823-4a82-9b64-41eb13e1a193 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '864d35f268cf45148756109759bb2743', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:53:02.266138 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705f3251-7823-4a82-9b64-41eb13e1a193 service cinder] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:53:02.266424 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705f3251-7823-4a82-9b64-41eb13e1a193 service cinder] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:53:02.266733 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705f3251-7823-4a82-9b64-41eb13e1a193 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:53:02.275763 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705f3251-7823-4a82-9b64-41eb13e1a193 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:53:02.277200 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705f3251-7823-4a82-9b64-41eb13e1a193 service cinder] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:53:02.286642 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1192/2408] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:53:02 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 10:53:02.362704 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e467b320-35c0-4a93-a165-2d5f20d9252b None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:53:02.372916 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e467b320-35c0-4a93-a165-2d5f20d9252b None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:53:02.376864 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b8522f-0c3e-42ee-86f2-ea75ba6be44e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '864d35f268cf45148756109759bb2743', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:53:02.377459 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b8522f-0c3e-42ee-86f2-ea75ba6be44e service cinder] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:53:02.377743 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b8522f-0c3e-42ee-86f2-ea75ba6be44e service cinder] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:53:02.378021 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b8522f-0c3e-42ee-86f2-ea75ba6be44e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:53:02.441564 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b8522f-0c3e-42ee-86f2-ea75ba6be44e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:53:02.443070 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b8522f-0c3e-42ee-86f2-ea75ba6be44e service cinder] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:53:02.452212 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1217/2409] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:53:02 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 10:53:11.285469 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705f3251-7823-4a82-9b64-41eb13e1a193 service cinder] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:53:11.324237 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:53:11.349581 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705f3251-7823-4a82-9b64-41eb13e1a193 service cinder] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:53:11.354088 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28848111-152f-4683-bb31-cf6be30d19c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:53:11.354769 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28848111-152f-4683-bb31-cf6be30d19c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:53:11.355054 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28848111-152f-4683-bb31-cf6be30d19c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:53:11.355329 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28848111-152f-4683-bb31-cf6be30d19c4 admin admin] PATH_INFO: `/v3/users/607eee6796384acda899c7c82f55ed21` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:53:11.356494 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28848111-152f-4683-bb31-cf6be30d19c4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=607eee6796384acda899c7c82f55ed21)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:53:11.358412 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28848111-152f-4683-bb31-cf6be30d19c4 admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:53:11.402512 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-28848111-152f-4683-bb31-cf6be30d19c4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '607eee6796384acda899c7c82f55ed21'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:53:11.409847 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-28848111-152f-4683-bb31-cf6be30d19c4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '607eee6796384acda899c7c82f55ed21'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:53:11.417197 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-28848111-152f-4683-bb31-cf6be30d19c4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '607eee6796384acda899c7c82f55ed21'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:53:11.417839 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-28848111-152f-4683-bb31-cf6be30d19c4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '607eee6796384acda899c7c82f55ed21'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:53:11.418490 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-28848111-152f-4683-bb31-cf6be30d19c4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '607eee6796384acda899c7c82f55ed21'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:53:11.435686 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1193/2410] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Jul 27 10:53:11 2022] DELETE /identity/v3/users/607eee6796384acda899c7c82f55ed21 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:53:11.440320 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d782261-cf3d-4980-b590-31ce92c50135 None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:53:11.440799 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d782261-cf3d-4980-b590-31ce92c50135 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:53:11.440897 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d782261-cf3d-4980-b590-31ce92c50135 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:53:11.460172 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-8d782261-cf3d-4980-b590-31ce92c50135 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:53:11.461380 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-8d782261-cf3d-4980-b590-31ce92c50135 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:53:11.520976 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1218/2411] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:53:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:53:11.531526 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28848111-152f-4683-bb31-cf6be30d19c4 admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:53:11.533015 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-28848111-152f-4683-bb31-cf6be30d19c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:53:11.589289 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28848111-152f-4683-bb31-cf6be30d19c4 admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:53:11.593590 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beae6745-d493-4a23-a314-491175688690 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7922d0a1a7c4e808d0b2a80dca0da40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:53:11.594208 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beae6745-d493-4a23-a314-491175688690 service neutron] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:53:11.594493 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beae6745-d493-4a23-a314-491175688690 service neutron] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:53:11.594802 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beae6745-d493-4a23-a314-491175688690 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:53:11.603605 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beae6745-d493-4a23-a314-491175688690 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:53:11.605026 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beae6745-d493-4a23-a314-491175688690 service neutron] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:53:11.614443 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1194/2412] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:53:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 10:53:11.879018 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d782261-cf3d-4980-b590-31ce92c50135 None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:53:11.880499 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-8d782261-cf3d-4980-b590-31ce92c50135 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:53:11.893531 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d782261-cf3d-4980-b590-31ce92c50135 None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:53:11.898237 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacaea29-adb6-4984-8333-8a65c1b5b307 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:53:11.898931 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacaea29-adb6-4984-8333-8a65c1b5b307 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:53:11.899212 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacaea29-adb6-4984-8333-8a65c1b5b307 admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:53:11.899485 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacaea29-adb6-4984-8333-8a65c1b5b307 admin admin] PATH_INFO: `/v3/projects/e1ec6223d4b5462ab93af12fa467c2ff` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:53:11.900660 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aacaea29-adb6-4984-8333-8a65c1b5b307 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e1ec6223d4b5462ab93af12fa467c2ff)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:53:11.903120 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aacaea29-adb6-4984-8333-8a65c1b5b307 admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:53:11.950831 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-aacaea29-adb6-4984-8333-8a65c1b5b307 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e1ec6223d4b5462ab93af12fa467c2ff', 'request_id': 'req-aacaea29-adb6-4984-8333-8a65c1b5b307'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:53:11.956941 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-aacaea29-adb6-4984-8333-8a65c1b5b307 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e1ec6223d4b5462ab93af12fa467c2ff', 'request_id': 'req-aacaea29-adb6-4984-8333-8a65c1b5b307'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:53:11.964710 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-aacaea29-adb6-4984-8333-8a65c1b5b307 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e1ec6223d4b5462ab93af12fa467c2ff', 'request_id': 'req-aacaea29-adb6-4984-8333-8a65c1b5b307'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:53:11.982466 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-aacaea29-adb6-4984-8333-8a65c1b5b307 admin admin] The token cache is being invalidate because project e1ec6223d4b5462ab93af12fa467c2ff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86827) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 10:53:11.982791 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-aacaea29-adb6-4984-8333-8a65c1b5b307 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:53:11.983433 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-aacaea29-adb6-4984-8333-8a65c1b5b307 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:53:11.984979 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1219/2413] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Jul 27 10:53:11 2022] DELETE /identity/v3/projects/e1ec6223d4b5462ab93af12fa467c2ff => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:53:11.988979 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beae6745-d493-4a23-a314-491175688690 service neutron] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:53:11.998648 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beae6745-d493-4a23-a314-491175688690 service neutron] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:53:12.049229 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:53:12.055610 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1098a995-4de4-435f-b303-5cb9b0d95afd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:53:12.056260 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1098a995-4de4-435f-b303-5cb9b0d95afd admin admin] REQUEST_METHOD: `DELETE` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:53:12.056919 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1098a995-4de4-435f-b303-5cb9b0d95afd admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:53:12.057196 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1098a995-4de4-435f-b303-5cb9b0d95afd admin admin] PATH_INFO: `/v3/users/784a47f327664c38986dd1cf2c238bbe` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:53:12.058377 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1098a995-4de4-435f-b303-5cb9b0d95afd admin admin] RBAC: Authorizing `identity:delete_user(user_id=784a47f327664c38986dd1cf2c238bbe)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:53:12.060806 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1098a995-4de4-435f-b303-5cb9b0d95afd admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:53:12.105552 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-1098a995-4de4-435f-b303-5cb9b0d95afd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '784a47f327664c38986dd1cf2c238bbe'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:53:12.113079 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-1098a995-4de4-435f-b303-5cb9b0d95afd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '784a47f327664c38986dd1cf2c238bbe'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:53:12.119686 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-1098a995-4de4-435f-b303-5cb9b0d95afd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '784a47f327664c38986dd1cf2c238bbe'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:53:12.120412 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-1098a995-4de4-435f-b303-5cb9b0d95afd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '784a47f327664c38986dd1cf2c238bbe'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:53:12.120993 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-1098a995-4de4-435f-b303-5cb9b0d95afd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '784a47f327664c38986dd1cf2c238bbe'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:53:12.139212 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1195/2414] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Jul 27 10:53:11 2022] DELETE /identity/v3/users/784a47f327664c38986dd1cf2c238bbe => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:53:12.399885 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacaea29-adb6-4984-8333-8a65c1b5b307 admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:53:12.401329 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-aacaea29-adb6-4984-8333-8a65c1b5b307 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:53:12.414247 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacaea29-adb6-4984-8333-8a65c1b5b307 admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:53:12.422632 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:53:12.474545 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c34e76e-187e-483c-b50f-af0c97817e0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:53:12.475238 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c34e76e-187e-483c-b50f-af0c97817e0f admin admin] REQUEST_METHOD: `DELETE` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:53:12.475472 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c34e76e-187e-483c-b50f-af0c97817e0f admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:53:12.475742 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c34e76e-187e-483c-b50f-af0c97817e0f admin admin] PATH_INFO: `/v3/projects/61846a14d822478882715e20cac82fa6` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:53:12.476896 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c34e76e-187e-483c-b50f-af0c97817e0f admin admin] RBAC: Authorizing `identity:delete_project(project_id=61846a14d822478882715e20cac82fa6)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:53:12.478812 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c34e76e-187e-483c-b50f-af0c97817e0f admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:53:12.525524 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-1c34e76e-187e-483c-b50f-af0c97817e0f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '61846a14d822478882715e20cac82fa6', 'request_id': 'req-1c34e76e-187e-483c-b50f-af0c97817e0f'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:53:12.531737 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-1c34e76e-187e-483c-b50f-af0c97817e0f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '61846a14d822478882715e20cac82fa6', 'request_id': 'req-1c34e76e-187e-483c-b50f-af0c97817e0f'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:53:12.539887 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-1c34e76e-187e-483c-b50f-af0c97817e0f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '61846a14d822478882715e20cac82fa6', 'request_id': 'req-1c34e76e-187e-483c-b50f-af0c97817e0f'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:53:12.550164 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-1c34e76e-187e-483c-b50f-af0c97817e0f admin admin] The token cache is being invalidate because project 61846a14d822478882715e20cac82fa6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86827) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 10:53:12.550447 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-1c34e76e-187e-483c-b50f-af0c97817e0f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:53:12.551092 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-1c34e76e-187e-483c-b50f-af0c97817e0f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:53:12.552583 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1220/2415] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Jul 27 10:53:12 2022] DELETE /identity/v3/projects/61846a14d822478882715e20cac82fa6 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:53:12.557718 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8899fcf-0e16-47f9-8e77-f56061a1e83f None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:53:12.558184 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8899fcf-0e16-47f9-8e77-f56061a1e83f None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:53:12.558311 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8899fcf-0e16-47f9-8e77-f56061a1e83f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:53:12.578496 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-e8899fcf-0e16-47f9-8e77-f56061a1e83f None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:53:12.579726 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-e8899fcf-0e16-47f9-8e77-f56061a1e83f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:53:12.638346 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1196/2416] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:53:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:53:12.645989 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13daa77d-f828-461c-87d5-fecc34772c0c None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:53:12.646273 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13daa77d-f828-461c-87d5-fecc34772c0c None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:53:12.646547 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13daa77d-f828-461c-87d5-fecc34772c0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:53:12.666030 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-13daa77d-f828-461c-87d5-fecc34772c0c None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:53:12.667243 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-13daa77d-f828-461c-87d5-fecc34772c0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:53:12.678532 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1221/2417] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:53:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:53:12.682904 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8899fcf-0e16-47f9-8e77-f56061a1e83f None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:53:12.692464 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8899fcf-0e16-47f9-8e77-f56061a1e83f None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:53:12.697241 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf76e031-36ce-474b-9796-71dcf23f68f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:53:12.698014 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf76e031-36ce-474b-9796-71dcf23f68f0 admin admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:53:12.698421 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf76e031-36ce-474b-9796-71dcf23f68f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:53:12.698756 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf76e031-36ce-474b-9796-71dcf23f68f0 admin admin] PATH_INFO: `/v3/domains` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:53:12.699455 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf76e031-36ce-474b-9796-71dcf23f68f0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86828) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:53:12.699759 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf76e031-36ce-474b-9796-71dcf23f68f0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:53:12.701350 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf76e031-36ce-474b-9796-71dcf23f68f0 admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:53:12.712904 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1197/2418] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Jul 27 10:53:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:53:12.717627 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13daa77d-f828-461c-87d5-fecc34772c0c None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:53:12.726989 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13daa77d-f828-461c-87d5-fecc34772c0c None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:53:12.732301 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aac7869-9712-41c5-8ccf-f14c7989a55c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:53:12.732907 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aac7869-9712-41c5-8ccf-f14c7989a55c admin admin] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:53:12.733189 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aac7869-9712-41c5-8ccf-f14c7989a55c admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:53:12.733463 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aac7869-9712-41c5-8ccf-f14c7989a55c admin admin] PATH_INFO: `/v3/projects` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:53:12.734133 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aac7869-9712-41c5-8ccf-f14c7989a55c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:53:12.736046 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aac7869-9712-41c5-8ccf-f14c7989a55c admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:53:12.764712 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1222/2419] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Jul 27 10:53:12 2022] POST /identity/v3/projects => generated 363 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 10:53:12.768964 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf76e031-36ce-474b-9796-71dcf23f68f0 admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:53:12.779408 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf76e031-36ce-474b-9796-71dcf23f68f0 admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:53:12.841748 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9e9d0a-597d-43d3-8425-1f3a3fb4db3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:53:12.842369 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9e9d0a-597d-43d3-8425-1f3a3fb4db3d admin admin] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:53:12.842685 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9e9d0a-597d-43d3-8425-1f3a3fb4db3d admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:53:12.842968 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9e9d0a-597d-43d3-8425-1f3a3fb4db3d admin admin] PATH_INFO: `/v3/users` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:53:12.843639 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9e9d0a-597d-43d3-8425-1f3a3fb4db3d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:53:12.845588 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9e9d0a-597d-43d3-8425-1f3a3fb4db3d admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:53:12.855325 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:53:12.868053 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1198/2420] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Jul 27 10:53:12 2022] POST /identity/v3/users => generated 282 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 10:53:12.872209 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aac7869-9712-41c5-8ccf-f14c7989a55c admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:53:12.882791 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aac7869-9712-41c5-8ccf-f14c7989a55c admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:53:12.888017 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29117006-7571-44bc-a788-a99c9568af92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:53:12.888626 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29117006-7571-44bc-a788-a99c9568af92 admin admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:53:12.888901 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29117006-7571-44bc-a788-a99c9568af92 admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:53:12.889183 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29117006-7571-44bc-a788-a99c9568af92 admin admin] PATH_INFO: `/v3/roles` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:53:12.889875 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29117006-7571-44bc-a788-a99c9568af92 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:53:12.891491 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29117006-7571-44bc-a788-a99c9568af92 admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:53:12.901293 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1223/2421] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 10:53:12 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 10:53:12.905939 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9e9d0a-597d-43d3-8425-1f3a3fb4db3d admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:53:12.915966 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9e9d0a-597d-43d3-8425-1f3a3fb4db3d admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:53:12.920586 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f362548-1daa-45c1-880a-afd72d72928e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:53:12.921201 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f362548-1daa-45c1-880a-afd72d72928e admin admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:53:12.921489 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f362548-1daa-45c1-880a-afd72d72928e admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:53:12.921768 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f362548-1daa-45c1-880a-afd72d72928e admin admin] PATH_INFO: `/v3/roles` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:53:12.922462 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f362548-1daa-45c1-880a-afd72d72928e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:53:12.924094 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f362548-1daa-45c1-880a-afd72d72928e admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:53:12.934513 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1199/2422] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 10:53:12 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 10:53:12.938552 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29117006-7571-44bc-a788-a99c9568af92 admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:53:12.949089 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29117006-7571-44bc-a788-a99c9568af92 admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:53:12.954286 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2096ad86-621b-433f-a1d9-3b695d9b2fea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:53:12.954992 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2096ad86-621b-433f-a1d9-3b695d9b2fea admin admin] REQUEST_METHOD: `PUT` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:53:12.955280 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2096ad86-621b-433f-a1d9-3b695d9b2fea admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:53:12.955559 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2096ad86-621b-433f-a1d9-3b695d9b2fea admin admin] PATH_INFO: `/v3/projects/3faf4859f6714a2b96003e73db9e8e18/users/1f78530aac474b11b9d8d1f20a674b91/roles/4c9fc0b3baed441da2126be41cc6b04a` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:53:12.971996 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2096ad86-621b-433f-a1d9-3b695d9b2fea admin admin] RBAC: Authorizing `identity:create_grant(project_id=3faf4859f6714a2b96003e73db9e8e18, user_id=1f78530aac474b11b9d8d1f20a674b91, role_id=4c9fc0b3baed441da2126be41cc6b04a)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:53:12.974683 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2096ad86-621b-433f-a1d9-3b695d9b2fea admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:53:12.987261 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1224/2423] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 10:53:12 2022] PUT /identity/v3/projects/3faf4859f6714a2b96003e73db9e8e18/users/1f78530aac474b11b9d8d1f20a674b91/roles/4c9fc0b3baed441da2126be41cc6b04a => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:53:12.998682 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932b4f63-4daa-4df5-a5c7-0b5a06f4a9f0 None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:53:12.998977 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932b4f63-4daa-4df5-a5c7-0b5a06f4a9f0 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:53:12.999518 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932b4f63-4daa-4df5-a5c7-0b5a06f4a9f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:53:13.027024 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-932b4f63-4daa-4df5-a5c7-0b5a06f4a9f0 None None] MFA Rules not processed for user `1f78530aac474b11b9d8d1f20a674b91`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:53:13.028863 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-932b4f63-4daa-4df5-a5c7-0b5a06f4a9f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:53:13.093840 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1200/2424] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:53:12 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:53:13.098387 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2096ad86-621b-433f-a1d9-3b695d9b2fea admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:53:13.139850 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:53:13.161582 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2096ad86-621b-433f-a1d9-3b695d9b2fea admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:53:13.166304 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed715aa-913e-4168-b2fb-7c0e3e969f99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:53:13.166951 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed715aa-913e-4168-b2fb-7c0e3e969f99 admin admin] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:53:13.167241 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed715aa-913e-4168-b2fb-7c0e3e969f99 admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:53:13.167517 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed715aa-913e-4168-b2fb-7c0e3e969f99 admin admin] PATH_INFO: `/v3/projects` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:53:13.168709 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed715aa-913e-4168-b2fb-7c0e3e969f99 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:53:13.170623 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed715aa-913e-4168-b2fb-7c0e3e969f99 admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:53:13.196379 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1225/2425] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Jul 27 10:53:13 2022] POST /identity/v3/projects => generated 363 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 10:53:13.201071 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932b4f63-4daa-4df5-a5c7-0b5a06f4a9f0 None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:53:13.211697 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932b4f63-4daa-4df5-a5c7-0b5a06f4a9f0 None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:53:13.267776 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5511a48-2842-41da-af39-cf1d05fbe6d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:53:13.268395 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5511a48-2842-41da-af39-cf1d05fbe6d9 admin admin] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:53:13.268676 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5511a48-2842-41da-af39-cf1d05fbe6d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:53:13.268951 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5511a48-2842-41da-af39-cf1d05fbe6d9 admin admin] PATH_INFO: `/v3/users` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:53:13.269620 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5511a48-2842-41da-af39-cf1d05fbe6d9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:53:13.271598 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5511a48-2842-41da-af39-cf1d05fbe6d9 admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:53:13.291988 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1201/2426] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Jul 27 10:53:13 2022] POST /identity/v3/users => generated 288 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 10:53:13.296064 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed715aa-913e-4168-b2fb-7c0e3e969f99 admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:53:13.306278 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed715aa-913e-4168-b2fb-7c0e3e969f99 admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:53:13.311519 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d66648-721a-4b23-821c-1cf100ff0344 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:53:13.312140 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d66648-721a-4b23-821c-1cf100ff0344 admin admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:53:13.312425 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d66648-721a-4b23-821c-1cf100ff0344 admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:53:13.312700 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d66648-721a-4b23-821c-1cf100ff0344 admin admin] PATH_INFO: `/v3/roles` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:53:13.313393 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d66648-721a-4b23-821c-1cf100ff0344 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:53:13.315033 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d66648-721a-4b23-821c-1cf100ff0344 admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:53:13.326070 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1226/2427] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 10:53:13 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 10:53:13.330259 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5511a48-2842-41da-af39-cf1d05fbe6d9 admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:53:13.340380 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5511a48-2842-41da-af39-cf1d05fbe6d9 admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:53:13.345103 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fa6170-9ae9-47dc-a947-452f05eae32b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:53:13.345711 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fa6170-9ae9-47dc-a947-452f05eae32b admin admin] REQUEST_METHOD: `PUT` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:53:13.345997 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fa6170-9ae9-47dc-a947-452f05eae32b admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:53:13.346274 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fa6170-9ae9-47dc-a947-452f05eae32b admin admin] PATH_INFO: `/v3/projects/41a87e0d65494af3b547aeb026b3641e/users/671be258a4804c1eb5c9df8efcaf5fac/roles/21156560eb6b436bb567a40b4976f3f4` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:53:13.362119 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fa6170-9ae9-47dc-a947-452f05eae32b admin admin] RBAC: Authorizing `identity:create_grant(project_id=41a87e0d65494af3b547aeb026b3641e, user_id=671be258a4804c1eb5c9df8efcaf5fac, role_id=21156560eb6b436bb567a40b4976f3f4)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:53:13.364808 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fa6170-9ae9-47dc-a947-452f05eae32b admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:53:13.378365 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1202/2428] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 10:53:13 2022] PUT /identity/v3/projects/41a87e0d65494af3b547aeb026b3641e/users/671be258a4804c1eb5c9df8efcaf5fac/roles/21156560eb6b436bb567a40b4976f3f4 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:53:13.382281 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d66648-721a-4b23-821c-1cf100ff0344 admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:53:13.392726 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d66648-721a-4b23-821c-1cf100ff0344 admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:53:13.447496 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef0f4ca-6a7e-43b9-b99c-48ac56017649 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:53:13.448114 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef0f4ca-6a7e-43b9-b99c-48ac56017649 admin admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:53:13.448397 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef0f4ca-6a7e-43b9-b99c-48ac56017649 admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:53:13.448674 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef0f4ca-6a7e-43b9-b99c-48ac56017649 admin admin] PATH_INFO: `/v3/roles` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:53:13.449454 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef0f4ca-6a7e-43b9-b99c-48ac56017649 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:53:13.451606 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef0f4ca-6a7e-43b9-b99c-48ac56017649 admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:53:13.461370 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1227/2429] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 10:53:13 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 10:53:13.465528 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fa6170-9ae9-47dc-a947-452f05eae32b admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:53:13.475556 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fa6170-9ae9-47dc-a947-452f05eae32b admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:53:13.480084 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668b323c-912c-4776-ae89-7e1f79a53bf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:53:13.480736 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668b323c-912c-4776-ae89-7e1f79a53bf9 admin admin] REQUEST_METHOD: `PUT` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:53:13.481014 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668b323c-912c-4776-ae89-7e1f79a53bf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:53:13.481281 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668b323c-912c-4776-ae89-7e1f79a53bf9 admin admin] PATH_INFO: `/v3/projects/41a87e0d65494af3b547aeb026b3641e/users/671be258a4804c1eb5c9df8efcaf5fac/roles/21156560eb6b436bb567a40b4976f3f4` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:53:13.483246 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668b323c-912c-4776-ae89-7e1f79a53bf9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=41a87e0d65494af3b547aeb026b3641e, user_id=671be258a4804c1eb5c9df8efcaf5fac, role_id=21156560eb6b436bb567a40b4976f3f4)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:53:13.485849 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668b323c-912c-4776-ae89-7e1f79a53bf9 admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:53:13.495727 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1203/2430] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 10:53:13 2022] PUT /identity/v3/projects/41a87e0d65494af3b547aeb026b3641e/users/671be258a4804c1eb5c9df8efcaf5fac/roles/21156560eb6b436bb567a40b4976f3f4 => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:53:13.506445 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9494a3-8614-4561-8f2e-956431436f8f None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:53:13.506985 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9494a3-8614-4561-8f2e-956431436f8f None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:53:13.507092 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9494a3-8614-4561-8f2e-956431436f8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:53:13.535041 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-8e9494a3-8614-4561-8f2e-956431436f8f None None] MFA Rules not processed for user `671be258a4804c1eb5c9df8efcaf5fac`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:53:13.536253 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-8e9494a3-8614-4561-8f2e-956431436f8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:53:13.602827 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1228/2431] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:53:13 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:53:13.614403 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668b323c-912c-4776-ae89-7e1f79a53bf9 admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:53:13.616146 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-668b323c-912c-4776-ae89-7e1f79a53bf9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:53:13.645047 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:53:13.675622 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668b323c-912c-4776-ae89-7e1f79a53bf9 admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:53:13.679211 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50696000-172f-437d-ae68-9b53fb507284 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '864d35f268cf45148756109759bb2743', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:53:13.679815 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50696000-172f-437d-ae68-9b53fb507284 service cinder] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:53:13.680089 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50696000-172f-437d-ae68-9b53fb507284 service cinder] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:53:13.680361 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50696000-172f-437d-ae68-9b53fb507284 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:53:13.689174 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50696000-172f-437d-ae68-9b53fb507284 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:53:13.690582 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50696000-172f-437d-ae68-9b53fb507284 service cinder] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:53:13.701597 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1204/2432] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:53:13 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 10:53:14.106433 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9494a3-8614-4561-8f2e-956431436f8f None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:53:14.140328 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:53:14.169388 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9494a3-8614-4561-8f2e-956431436f8f None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:53:14.173798 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945d557b-d7ec-4fc3-8e4a-36689c057962 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:53:14.174452 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945d557b-d7ec-4fc3-8e4a-36689c057962 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:53:14.174765 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945d557b-d7ec-4fc3-8e4a-36689c057962 admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:53:14.175041 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945d557b-d7ec-4fc3-8e4a-36689c057962 admin admin] PATH_INFO: `/v3/users/1f78530aac474b11b9d8d1f20a674b91` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:53:14.176187 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945d557b-d7ec-4fc3-8e4a-36689c057962 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1f78530aac474b11b9d8d1f20a674b91)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:53:14.178087 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945d557b-d7ec-4fc3-8e4a-36689c057962 admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:53:14.222519 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-945d557b-d7ec-4fc3-8e4a-36689c057962 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1f78530aac474b11b9d8d1f20a674b91'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:53:14.229531 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-945d557b-d7ec-4fc3-8e4a-36689c057962 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1f78530aac474b11b9d8d1f20a674b91'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:53:14.237044 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-945d557b-d7ec-4fc3-8e4a-36689c057962 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1f78530aac474b11b9d8d1f20a674b91'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:53:14.237638 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-945d557b-d7ec-4fc3-8e4a-36689c057962 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1f78530aac474b11b9d8d1f20a674b91'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:53:14.238241 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-945d557b-d7ec-4fc3-8e4a-36689c057962 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1f78530aac474b11b9d8d1f20a674b91'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:53:14.255718 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1229/2433] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Jul 27 10:53:14 2022] DELETE /identity/v3/users/1f78530aac474b11b9d8d1f20a674b91 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:53:14.260230 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb64306-c8a4-4861-aa33-c7ec13c59a33 None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:53:14.260527 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb64306-c8a4-4861-aa33-c7ec13c59a33 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:53:14.260808 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb64306-c8a4-4861-aa33-c7ec13c59a33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:53:14.279729 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-bfb64306-c8a4-4861-aa33-c7ec13c59a33 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:53:14.280926 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-bfb64306-c8a4-4861-aa33-c7ec13c59a33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:53:14.339160 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1205/2434] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:53:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:53:14.348866 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945d557b-d7ec-4fc3-8e4a-36689c057962 admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:53:14.350361 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-945d557b-d7ec-4fc3-8e4a-36689c057962 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:53:14.407626 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945d557b-d7ec-4fc3-8e4a-36689c057962 admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:53:14.411196 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80e975c-6268-4a26-ba77-198347ff6229 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7922d0a1a7c4e808d0b2a80dca0da40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:53:14.411810 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80e975c-6268-4a26-ba77-198347ff6229 service neutron] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:53:14.412095 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80e975c-6268-4a26-ba77-198347ff6229 service neutron] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:53:14.412446 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80e975c-6268-4a26-ba77-198347ff6229 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:53:14.421839 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80e975c-6268-4a26-ba77-198347ff6229 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:53:14.423378 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80e975c-6268-4a26-ba77-198347ff6229 service neutron] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:53:14.432840 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1230/2435] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:53:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 10:53:14.698055 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb64306-c8a4-4861-aa33-c7ec13c59a33 None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:53:14.699932 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-bfb64306-c8a4-4861-aa33-c7ec13c59a33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:53:14.712244 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb64306-c8a4-4861-aa33-c7ec13c59a33 None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:53:14.717591 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa85a89-f96d-402a-89a5-6d9085220a72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:53:14.718318 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa85a89-f96d-402a-89a5-6d9085220a72 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:53:14.718611 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa85a89-f96d-402a-89a5-6d9085220a72 admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:53:14.718888 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa85a89-f96d-402a-89a5-6d9085220a72 admin admin] PATH_INFO: `/v3/projects/3faf4859f6714a2b96003e73db9e8e18` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:53:14.720046 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa85a89-f96d-402a-89a5-6d9085220a72 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3faf4859f6714a2b96003e73db9e8e18)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:53:14.721949 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa85a89-f96d-402a-89a5-6d9085220a72 admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:53:14.768973 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-afa85a89-f96d-402a-89a5-6d9085220a72 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3faf4859f6714a2b96003e73db9e8e18', 'request_id': 'req-afa85a89-f96d-402a-89a5-6d9085220a72'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:53:14.775656 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-afa85a89-f96d-402a-89a5-6d9085220a72 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3faf4859f6714a2b96003e73db9e8e18', 'request_id': 'req-afa85a89-f96d-402a-89a5-6d9085220a72'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:53:14.783624 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-afa85a89-f96d-402a-89a5-6d9085220a72 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3faf4859f6714a2b96003e73db9e8e18', 'request_id': 'req-afa85a89-f96d-402a-89a5-6d9085220a72'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:53:14.794428 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-afa85a89-f96d-402a-89a5-6d9085220a72 admin admin] The token cache is being invalidate because project 3faf4859f6714a2b96003e73db9e8e18 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86828) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 10:53:14.794752 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-afa85a89-f96d-402a-89a5-6d9085220a72 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:53:14.795380 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-afa85a89-f96d-402a-89a5-6d9085220a72 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:53:14.796889 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1206/2436] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Jul 27 10:53:14 2022] DELETE /identity/v3/projects/3faf4859f6714a2b96003e73db9e8e18 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:53:14.800989 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80e975c-6268-4a26-ba77-198347ff6229 service neutron] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:53:14.810710 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80e975c-6268-4a26-ba77-198347ff6229 service neutron] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:53:14.860069 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:53:14.867735 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab03375-b3c6-4380-8a48-cefb29505e3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:53:14.868495 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab03375-b3c6-4380-8a48-cefb29505e3c admin admin] REQUEST_METHOD: `DELETE` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:53:14.868948 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab03375-b3c6-4380-8a48-cefb29505e3c admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:53:14.869384 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab03375-b3c6-4380-8a48-cefb29505e3c admin admin] PATH_INFO: `/v3/users/671be258a4804c1eb5c9df8efcaf5fac` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:53:14.870682 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab03375-b3c6-4380-8a48-cefb29505e3c admin admin] RBAC: Authorizing `identity:delete_user(user_id=671be258a4804c1eb5c9df8efcaf5fac)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:53:14.872658 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab03375-b3c6-4380-8a48-cefb29505e3c admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:53:14.917907 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-cab03375-b3c6-4380-8a48-cefb29505e3c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '671be258a4804c1eb5c9df8efcaf5fac'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:53:14.925271 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-cab03375-b3c6-4380-8a48-cefb29505e3c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '671be258a4804c1eb5c9df8efcaf5fac'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:53:14.931919 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-cab03375-b3c6-4380-8a48-cefb29505e3c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '671be258a4804c1eb5c9df8efcaf5fac'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:53:14.932547 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-cab03375-b3c6-4380-8a48-cefb29505e3c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '671be258a4804c1eb5c9df8efcaf5fac'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:53:14.933154 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-cab03375-b3c6-4380-8a48-cefb29505e3c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '671be258a4804c1eb5c9df8efcaf5fac'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:53:14.950269 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1231/2437] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Jul 27 10:53:14 2022] DELETE /identity/v3/users/671be258a4804c1eb5c9df8efcaf5fac => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:53:15.214160 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa85a89-f96d-402a-89a5-6d9085220a72 admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:53:15.216172 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-afa85a89-f96d-402a-89a5-6d9085220a72 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:53:15.232219 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa85a89-f96d-402a-89a5-6d9085220a72 admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:53:15.281081 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:53:15.291933 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7fcd25-18dd-41dc-b8c1-edf35481eead admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:53:15.292599 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7fcd25-18dd-41dc-b8c1-edf35481eead admin admin] REQUEST_METHOD: `DELETE` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:53:15.292878 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7fcd25-18dd-41dc-b8c1-edf35481eead admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:53:15.293151 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7fcd25-18dd-41dc-b8c1-edf35481eead admin admin] PATH_INFO: `/v3/projects/41a87e0d65494af3b547aeb026b3641e` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:53:15.294283 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7fcd25-18dd-41dc-b8c1-edf35481eead admin admin] RBAC: Authorizing `identity:delete_project(project_id=41a87e0d65494af3b547aeb026b3641e)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:53:15.296221 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7fcd25-18dd-41dc-b8c1-edf35481eead admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:53:15.341101 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-8d7fcd25-18dd-41dc-b8c1-edf35481eead admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '41a87e0d65494af3b547aeb026b3641e', 'request_id': 'req-8d7fcd25-18dd-41dc-b8c1-edf35481eead'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:53:15.347019 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-8d7fcd25-18dd-41dc-b8c1-edf35481eead admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '41a87e0d65494af3b547aeb026b3641e', 'request_id': 'req-8d7fcd25-18dd-41dc-b8c1-edf35481eead'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:53:15.354122 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-8d7fcd25-18dd-41dc-b8c1-edf35481eead admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '41a87e0d65494af3b547aeb026b3641e', 'request_id': 'req-8d7fcd25-18dd-41dc-b8c1-edf35481eead'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:53:15.364697 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-8d7fcd25-18dd-41dc-b8c1-edf35481eead admin admin] The token cache is being invalidate because project 41a87e0d65494af3b547aeb026b3641e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86828) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 10:53:15.364989 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-8d7fcd25-18dd-41dc-b8c1-edf35481eead admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:53:15.365621 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-8d7fcd25-18dd-41dc-b8c1-edf35481eead admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:53:15.367162 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1207/2438] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Jul 27 10:53:15 2022] DELETE /identity/v3/projects/41a87e0d65494af3b547aeb026b3641e => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:53:15.372162 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f256ec09-d184-4c1e-afae-a181b7021069 None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:53:15.372456 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f256ec09-d184-4c1e-afae-a181b7021069 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:53:15.372767 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f256ec09-d184-4c1e-afae-a181b7021069 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:53:15.392568 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-f256ec09-d184-4c1e-afae-a181b7021069 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:53:15.393769 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-f256ec09-d184-4c1e-afae-a181b7021069 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:53:15.452343 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1232/2439] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:53:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:53:15.461446 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e032906-48a5-4a51-94a2-cb9d8a6fd8e5 None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:53:15.461672 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e032906-48a5-4a51-94a2-cb9d8a6fd8e5 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:53:15.461946 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e032906-48a5-4a51-94a2-cb9d8a6fd8e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:53:15.481977 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-9e032906-48a5-4a51-94a2-cb9d8a6fd8e5 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:53:15.483215 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-9e032906-48a5-4a51-94a2-cb9d8a6fd8e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:53:15.491056 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1208/2440] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:53:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:53:15.495413 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f256ec09-d184-4c1e-afae-a181b7021069 None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:53:15.505806 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f256ec09-d184-4c1e-afae-a181b7021069 None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:53:15.510314 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703d1822-e30f-41da-870f-f1914b3c0d1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:53:15.510975 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703d1822-e30f-41da-870f-f1914b3c0d1c admin admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:53:15.511258 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703d1822-e30f-41da-870f-f1914b3c0d1c admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:53:15.511530 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703d1822-e30f-41da-870f-f1914b3c0d1c admin admin] PATH_INFO: `/v3/domains` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:53:15.512763 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703d1822-e30f-41da-870f-f1914b3c0d1c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86827) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:53:15.513065 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703d1822-e30f-41da-870f-f1914b3c0d1c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:53:15.514686 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703d1822-e30f-41da-870f-f1914b3c0d1c admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:53:15.526013 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1233/2441] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Jul 27 10:53:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:53:15.530274 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e032906-48a5-4a51-94a2-cb9d8a6fd8e5 None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:53:15.539576 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e032906-48a5-4a51-94a2-cb9d8a6fd8e5 None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:53:15.544256 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e0edd8-505e-43d1-91bc-cbe517aa3d36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:53:15.544866 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e0edd8-505e-43d1-91bc-cbe517aa3d36 admin admin] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:53:15.545154 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e0edd8-505e-43d1-91bc-cbe517aa3d36 admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:53:15.545420 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e0edd8-505e-43d1-91bc-cbe517aa3d36 admin admin] PATH_INFO: `/v3/projects` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:53:15.546074 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e0edd8-505e-43d1-91bc-cbe517aa3d36 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:53:15.548023 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e0edd8-505e-43d1-91bc-cbe517aa3d36 admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:53:15.574829 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1209/2442] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Jul 27 10:53:15 2022] POST /identity/v3/projects => generated 363 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 10:53:15.579288 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703d1822-e30f-41da-870f-f1914b3c0d1c admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:53:15.589247 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703d1822-e30f-41da-870f-f1914b3c0d1c admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:53:15.649128 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55548c34-a2f0-4bdf-b299-b514df80e0f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:53:15.649744 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55548c34-a2f0-4bdf-b299-b514df80e0f0 admin admin] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:53:15.650030 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55548c34-a2f0-4bdf-b299-b514df80e0f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:53:15.650307 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55548c34-a2f0-4bdf-b299-b514df80e0f0 admin admin] PATH_INFO: `/v3/users` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:53:15.651024 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55548c34-a2f0-4bdf-b299-b514df80e0f0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:53:15.652957 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55548c34-a2f0-4bdf-b299-b514df80e0f0 admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:53:15.661717 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:53:15.673967 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1234/2443] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Jul 27 10:53:15 2022] POST /identity/v3/users => generated 282 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 10:53:15.678130 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e0edd8-505e-43d1-91bc-cbe517aa3d36 admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:53:15.688904 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e0edd8-505e-43d1-91bc-cbe517aa3d36 admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:53:15.693602 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6838abeb-c6c3-4259-b490-1ceba883c37f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:53:15.694214 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6838abeb-c6c3-4259-b490-1ceba883c37f admin admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:53:15.694497 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6838abeb-c6c3-4259-b490-1ceba883c37f admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:53:15.694811 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6838abeb-c6c3-4259-b490-1ceba883c37f admin admin] PATH_INFO: `/v3/roles` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:53:15.695532 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6838abeb-c6c3-4259-b490-1ceba883c37f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:53:15.697301 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6838abeb-c6c3-4259-b490-1ceba883c37f admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:53:15.708584 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1210/2444] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 10:53:15 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 10:53:15.712856 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55548c34-a2f0-4bdf-b299-b514df80e0f0 admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:53:15.723238 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55548c34-a2f0-4bdf-b299-b514df80e0f0 admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:53:15.733205 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162c6698-0c09-4e63-b406-24f067cf9b71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:53:15.733849 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162c6698-0c09-4e63-b406-24f067cf9b71 admin admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:53:15.734127 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162c6698-0c09-4e63-b406-24f067cf9b71 admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:53:15.734399 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162c6698-0c09-4e63-b406-24f067cf9b71 admin admin] PATH_INFO: `/v3/roles` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:53:15.735199 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162c6698-0c09-4e63-b406-24f067cf9b71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:53:15.736806 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162c6698-0c09-4e63-b406-24f067cf9b71 admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:53:15.748271 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1235/2445] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 10:53:15 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 10:53:15.752968 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6838abeb-c6c3-4259-b490-1ceba883c37f admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:53:15.763795 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6838abeb-c6c3-4259-b490-1ceba883c37f admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:53:15.768475 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c4e16c-e65a-43f5-9709-1c958468a10a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:53:15.769138 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c4e16c-e65a-43f5-9709-1c958468a10a admin admin] REQUEST_METHOD: `PUT` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:53:15.769425 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c4e16c-e65a-43f5-9709-1c958468a10a admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:53:15.769705 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c4e16c-e65a-43f5-9709-1c958468a10a admin admin] PATH_INFO: `/v3/projects/75867705261d484e80b353201f7ce226/users/d0312f1f1d284ccf9dc018875212cd51/roles/4c9fc0b3baed441da2126be41cc6b04a` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:53:15.784111 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c4e16c-e65a-43f5-9709-1c958468a10a admin admin] RBAC: Authorizing `identity:create_grant(project_id=75867705261d484e80b353201f7ce226, user_id=d0312f1f1d284ccf9dc018875212cd51, role_id=4c9fc0b3baed441da2126be41cc6b04a)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:53:15.786798 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c4e16c-e65a-43f5-9709-1c958468a10a admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:53:15.800138 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1211/2446] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 10:53:15 2022] PUT /identity/v3/projects/75867705261d484e80b353201f7ce226/users/d0312f1f1d284ccf9dc018875212cd51/roles/4c9fc0b3baed441da2126be41cc6b04a => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 10:53:15.811185 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c648cd50-380b-40b6-b77a-6ffbd34811ea None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:53:15.811679 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c648cd50-380b-40b6-b77a-6ffbd34811ea None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:53:15.811779 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c648cd50-380b-40b6-b77a-6ffbd34811ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:53:15.839819 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-c648cd50-380b-40b6-b77a-6ffbd34811ea None None] MFA Rules not processed for user `d0312f1f1d284ccf9dc018875212cd51`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:53:15.841059 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-c648cd50-380b-40b6-b77a-6ffbd34811ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:53:15.904479 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1236/2447] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:53:15 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:53:15.909085 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c4e16c-e65a-43f5-9709-1c958468a10a admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:53:15.970417 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c4e16c-e65a-43f5-9709-1c958468a10a admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:53:15.975059 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0039ef-90ed-4d09-92d7-19d6df840acd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:53:15.975696 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0039ef-90ed-4d09-92d7-19d6df840acd admin admin] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:53:15.975967 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0039ef-90ed-4d09-92d7-19d6df840acd admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:53:15.976248 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0039ef-90ed-4d09-92d7-19d6df840acd admin admin] PATH_INFO: `/v3/projects` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:53:15.976930 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0039ef-90ed-4d09-92d7-19d6df840acd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:53:15.978898 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0039ef-90ed-4d09-92d7-19d6df840acd admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:53:16.005625 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1212/2448] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Jul 27 10:53:15 2022] POST /identity/v3/projects => generated 365 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 10:53:16.009689 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c648cd50-380b-40b6-b77a-6ffbd34811ea None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:53:16.020616 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c648cd50-380b-40b6-b77a-6ffbd34811ea None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:53:16.076335 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431ba610-c0a3-42dd-9282-14e86396cf51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:53:16.076950 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431ba610-c0a3-42dd-9282-14e86396cf51 admin admin] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:53:16.077231 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431ba610-c0a3-42dd-9282-14e86396cf51 admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:53:16.077507 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431ba610-c0a3-42dd-9282-14e86396cf51 admin admin] PATH_INFO: `/v3/users` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:53:16.078176 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431ba610-c0a3-42dd-9282-14e86396cf51 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:53:16.080147 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431ba610-c0a3-42dd-9282-14e86396cf51 admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:53:16.101502 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1237/2449] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Jul 27 10:53:16 2022] POST /identity/v3/users => generated 289 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 10:53:16.105511 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0039ef-90ed-4d09-92d7-19d6df840acd admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:53:16.111546 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:53:16.117928 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0039ef-90ed-4d09-92d7-19d6df840acd admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:53:16.124011 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44568a9-81e1-410e-b912-e83418c83ce3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:53:16.124899 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44568a9-81e1-410e-b912-e83418c83ce3 admin admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:53:16.125187 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44568a9-81e1-410e-b912-e83418c83ce3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:53:16.125466 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44568a9-81e1-410e-b912-e83418c83ce3 admin admin] PATH_INFO: `/v3/roles` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:53:16.126160 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44568a9-81e1-410e-b912-e83418c83ce3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:53:16.127806 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44568a9-81e1-410e-b912-e83418c83ce3 admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:53:16.139470 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1213/2450] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 10:53:16 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 10:53:16.143511 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431ba610-c0a3-42dd-9282-14e86396cf51 admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:53:16.153439 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431ba610-c0a3-42dd-9282-14e86396cf51 admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:53:16.157965 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f4ff4e-cff3-4f6b-8697-241c1afe364e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:53:16.158643 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f4ff4e-cff3-4f6b-8697-241c1afe364e admin admin] REQUEST_METHOD: `PUT` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:53:16.158928 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f4ff4e-cff3-4f6b-8697-241c1afe364e admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:53:16.159201 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f4ff4e-cff3-4f6b-8697-241c1afe364e admin admin] PATH_INFO: `/v3/projects/8e90721f2b4847be95c19217190b6273/users/c45c0a7451d949d89efe0bcd857586da/roles/21156560eb6b436bb567a40b4976f3f4` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:53:16.173915 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f4ff4e-cff3-4f6b-8697-241c1afe364e admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e90721f2b4847be95c19217190b6273, user_id=c45c0a7451d949d89efe0bcd857586da, role_id=21156560eb6b436bb567a40b4976f3f4)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:53:16.176569 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f4ff4e-cff3-4f6b-8697-241c1afe364e admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:53:16.190269 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1238/2451] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 10:53:16 2022] PUT /identity/v3/projects/8e90721f2b4847be95c19217190b6273/users/c45c0a7451d949d89efe0bcd857586da/roles/21156560eb6b436bb567a40b4976f3f4 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 10:53:16.194286 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44568a9-81e1-410e-b912-e83418c83ce3 admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:53:16.204417 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44568a9-81e1-410e-b912-e83418c83ce3 admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:53:16.260155 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc7fc08-4199-4999-b2cf-3fd488df426d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:53:16.260697 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc7fc08-4199-4999-b2cf-3fd488df426d admin admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:53:16.260979 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc7fc08-4199-4999-b2cf-3fd488df426d admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:53:16.261250 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc7fc08-4199-4999-b2cf-3fd488df426d admin admin] PATH_INFO: `/v3/roles` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:53:16.261919 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc7fc08-4199-4999-b2cf-3fd488df426d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:53:16.263579 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc7fc08-4199-4999-b2cf-3fd488df426d admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:53:16.273188 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1214/2452] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 10:53:16 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 10:53:16.277264 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f4ff4e-cff3-4f6b-8697-241c1afe364e admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:53:16.288095 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f4ff4e-cff3-4f6b-8697-241c1afe364e admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:53:16.292776 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61fd83a-87b6-41e3-ad2d-91600b77eaf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:53:16.294035 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61fd83a-87b6-41e3-ad2d-91600b77eaf1 admin admin] REQUEST_METHOD: `PUT` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:53:16.294323 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61fd83a-87b6-41e3-ad2d-91600b77eaf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:53:16.294619 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61fd83a-87b6-41e3-ad2d-91600b77eaf1 admin admin] PATH_INFO: `/v3/projects/8e90721f2b4847be95c19217190b6273/users/c45c0a7451d949d89efe0bcd857586da/roles/21156560eb6b436bb567a40b4976f3f4` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:53:16.296584 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61fd83a-87b6-41e3-ad2d-91600b77eaf1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e90721f2b4847be95c19217190b6273, user_id=c45c0a7451d949d89efe0bcd857586da, role_id=21156560eb6b436bb567a40b4976f3f4)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:53:16.299261 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61fd83a-87b6-41e3-ad2d-91600b77eaf1 admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:53:16.309753 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1239/2453] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 10:53:16 2022] PUT /identity/v3/projects/8e90721f2b4847be95c19217190b6273/users/c45c0a7451d949d89efe0bcd857586da/roles/21156560eb6b436bb567a40b4976f3f4 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:53:16.319867 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4590f0da-5c92-441f-a99b-c093d66fc620 None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:53:16.320155 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4590f0da-5c92-441f-a99b-c093d66fc620 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:53:16.320429 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4590f0da-5c92-441f-a99b-c093d66fc620 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:53:16.347925 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-4590f0da-5c92-441f-a99b-c093d66fc620 None None] MFA Rules not processed for user `c45c0a7451d949d89efe0bcd857586da`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:53:16.349109 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-4590f0da-5c92-441f-a99b-c093d66fc620 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:53:16.415677 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1215/2454] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:53:16 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 10:53:16.426995 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61fd83a-87b6-41e3-ad2d-91600b77eaf1 admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:53:16.428715 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-f61fd83a-87b6-41e3-ad2d-91600b77eaf1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:53:16.455886 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:INSERT=7,keystone:SELECT=93 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:53:16.487592 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61fd83a-87b6-41e3-ad2d-91600b77eaf1 admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:53:16.491227 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a41a574-6ff8-40fe-9c5d-28899410bf6f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '864d35f268cf45148756109759bb2743', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:53:16.491840 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a41a574-6ff8-40fe-9c5d-28899410bf6f service cinder] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:53:16.492122 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a41a574-6ff8-40fe-9c5d-28899410bf6f service cinder] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:53:16.492395 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a41a574-6ff8-40fe-9c5d-28899410bf6f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:53:16.502009 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a41a574-6ff8-40fe-9c5d-28899410bf6f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:53:16.503467 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a41a574-6ff8-40fe-9c5d-28899410bf6f service cinder] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:53:16.513497 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1240/2455] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:53:16 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 10:53:16.587140 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4590f0da-5c92-441f-a99b-c093d66fc620 None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:53:16.596689 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4590f0da-5c92-441f-a99b-c093d66fc620 None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:53:16.600391 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9520e145-d8af-4c5f-b8db-e123c07b75d7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '864d35f268cf45148756109759bb2743', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:53:16.601006 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9520e145-d8af-4c5f-b8db-e123c07b75d7 service cinder] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:53:16.601288 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9520e145-d8af-4c5f-b8db-e123c07b75d7 service cinder] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:53:16.601599 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9520e145-d8af-4c5f-b8db-e123c07b75d7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:53:16.658304 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9520e145-d8af-4c5f-b8db-e123c07b75d7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:53:16.659790 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9520e145-d8af-4c5f-b8db-e123c07b75d7 service cinder] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:53:16.668849 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1216/2456] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:53:16 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 10:53:18.798015 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a41a574-6ff8-40fe-9c5d-28899410bf6f service cinder] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:53:18.799540 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-0a41a574-6ff8-40fe-9c5d-28899410bf6f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:53:18.872462 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a41a574-6ff8-40fe-9c5d-28899410bf6f service cinder] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:53:18.878093 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce265ad2-cdaa-4f9e-846a-1036252fb322 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e059d8b6c28c4fcba925d05d7c98e4ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:53:18.878840 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce265ad2-cdaa-4f9e-846a-1036252fb322 service nova] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:53:18.879125 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce265ad2-cdaa-4f9e-846a-1036252fb322 service nova] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:53:18.879402 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce265ad2-cdaa-4f9e-846a-1036252fb322 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:53:18.881106 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-ce265ad2-cdaa-4f9e-846a-1036252fb322 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:53:18.938889 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce265ad2-cdaa-4f9e-846a-1036252fb322 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:53:18.940495 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce265ad2-cdaa-4f9e-846a-1036252fb322 service nova] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:53:18.950664 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1241/2457] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:53:18 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 10:53:25.941820 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9520e145-d8af-4c5f-b8db-e123c07b75d7 service cinder] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:53:25.954572 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:53:26.006263 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9520e145-d8af-4c5f-b8db-e123c07b75d7 service cinder] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:53:26.010846 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5826aa2-51de-4a92-b12d-2bcbdc45cb3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:53:26.011487 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5826aa2-51de-4a92-b12d-2bcbdc45cb3f admin admin] REQUEST_METHOD: `DELETE` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:53:26.011768 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5826aa2-51de-4a92-b12d-2bcbdc45cb3f admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:53:26.012042 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5826aa2-51de-4a92-b12d-2bcbdc45cb3f admin admin] PATH_INFO: `/v3/users/d0312f1f1d284ccf9dc018875212cd51` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:53:26.013199 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5826aa2-51de-4a92-b12d-2bcbdc45cb3f admin admin] RBAC: Authorizing `identity:delete_user(user_id=d0312f1f1d284ccf9dc018875212cd51)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:53:26.015169 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5826aa2-51de-4a92-b12d-2bcbdc45cb3f admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:53:26.064223 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-e5826aa2-51de-4a92-b12d-2bcbdc45cb3f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd0312f1f1d284ccf9dc018875212cd51'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:53:26.071091 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-e5826aa2-51de-4a92-b12d-2bcbdc45cb3f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd0312f1f1d284ccf9dc018875212cd51'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:53:26.078362 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-e5826aa2-51de-4a92-b12d-2bcbdc45cb3f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd0312f1f1d284ccf9dc018875212cd51'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:53:26.079027 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-e5826aa2-51de-4a92-b12d-2bcbdc45cb3f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd0312f1f1d284ccf9dc018875212cd51'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:53:26.079635 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-e5826aa2-51de-4a92-b12d-2bcbdc45cb3f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd0312f1f1d284ccf9dc018875212cd51'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:53:26.096974 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1217/2458] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Jul 27 10:53:25 2022] DELETE /identity/v3/users/d0312f1f1d284ccf9dc018875212cd51 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:53:26.101506 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc2cc53-c08b-4fa5-b087-ca61dc804c64 None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:53:26.102006 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc2cc53-c08b-4fa5-b087-ca61dc804c64 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:53:26.102115 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc2cc53-c08b-4fa5-b087-ca61dc804c64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:53:26.121353 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-5bc2cc53-c08b-4fa5-b087-ca61dc804c64 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:53:26.122555 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-5bc2cc53-c08b-4fa5-b087-ca61dc804c64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:53:26.182010 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1242/2459] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:53:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:53:26.191723 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5826aa2-51de-4a92-b12d-2bcbdc45cb3f admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:53:26.193432 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-e5826aa2-51de-4a92-b12d-2bcbdc45cb3f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:53:26.249959 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5826aa2-51de-4a92-b12d-2bcbdc45cb3f admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:53:26.253647 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676931b9-9bca-47b0-b1bd-5a2b81269e63 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7922d0a1a7c4e808d0b2a80dca0da40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:53:26.254267 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676931b9-9bca-47b0-b1bd-5a2b81269e63 service neutron] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:53:26.254548 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676931b9-9bca-47b0-b1bd-5a2b81269e63 service neutron] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:53:26.254852 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676931b9-9bca-47b0-b1bd-5a2b81269e63 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:53:26.263914 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676931b9-9bca-47b0-b1bd-5a2b81269e63 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:53:26.265338 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676931b9-9bca-47b0-b1bd-5a2b81269e63 service neutron] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:53:26.274674 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1218/2460] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:53:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 10:53:26.540724 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc2cc53-c08b-4fa5-b087-ca61dc804c64 None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:53:26.542372 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-5bc2cc53-c08b-4fa5-b087-ca61dc804c64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:53:26.549608 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:53:26.555246 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc2cc53-c08b-4fa5-b087-ca61dc804c64 None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:53:26.561236 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52010ad-0f23-4fa3-bf69-315bf57db78c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:53:26.561885 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52010ad-0f23-4fa3-bf69-315bf57db78c admin admin] REQUEST_METHOD: `DELETE` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:53:26.562193 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52010ad-0f23-4fa3-bf69-315bf57db78c admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:53:26.562582 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52010ad-0f23-4fa3-bf69-315bf57db78c admin admin] PATH_INFO: `/v3/projects/75867705261d484e80b353201f7ce226` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:53:26.563803 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52010ad-0f23-4fa3-bf69-315bf57db78c admin admin] RBAC: Authorizing `identity:delete_project(project_id=75867705261d484e80b353201f7ce226)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:53:26.565706 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52010ad-0f23-4fa3-bf69-315bf57db78c admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:53:26.613724 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-e52010ad-0f23-4fa3-bf69-315bf57db78c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '75867705261d484e80b353201f7ce226', 'request_id': 'req-e52010ad-0f23-4fa3-bf69-315bf57db78c'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:53:26.619819 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-e52010ad-0f23-4fa3-bf69-315bf57db78c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '75867705261d484e80b353201f7ce226', 'request_id': 'req-e52010ad-0f23-4fa3-bf69-315bf57db78c'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:53:26.627440 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-e52010ad-0f23-4fa3-bf69-315bf57db78c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '75867705261d484e80b353201f7ce226', 'request_id': 'req-e52010ad-0f23-4fa3-bf69-315bf57db78c'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:53:26.638488 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-e52010ad-0f23-4fa3-bf69-315bf57db78c admin admin] The token cache is being invalidate because project 75867705261d484e80b353201f7ce226 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86827) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 10:53:26.638806 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-e52010ad-0f23-4fa3-bf69-315bf57db78c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:53:26.639459 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-e52010ad-0f23-4fa3-bf69-315bf57db78c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:53:26.641023 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1243/2461] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Jul 27 10:53:26 2022] DELETE /identity/v3/projects/75867705261d484e80b353201f7ce226 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 10:53:26.645153 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676931b9-9bca-47b0-b1bd-5a2b81269e63 service neutron] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:53:26.655427 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676931b9-9bca-47b0-b1bd-5a2b81269e63 service neutron] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:53:26.687365 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:53:26.722931 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae34aaf-3bab-4b09-b521-50e609166696 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:53:26.723579 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae34aaf-3bab-4b09-b521-50e609166696 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:53:26.723860 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae34aaf-3bab-4b09-b521-50e609166696 admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:53:26.724135 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae34aaf-3bab-4b09-b521-50e609166696 admin admin] PATH_INFO: `/v3/users/c45c0a7451d949d89efe0bcd857586da` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:53:26.725454 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae34aaf-3bab-4b09-b521-50e609166696 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c45c0a7451d949d89efe0bcd857586da)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:53:26.727431 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae34aaf-3bab-4b09-b521-50e609166696 admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:53:26.772740 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-9ae34aaf-3bab-4b09-b521-50e609166696 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c45c0a7451d949d89efe0bcd857586da'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:53:26.780334 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-9ae34aaf-3bab-4b09-b521-50e609166696 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c45c0a7451d949d89efe0bcd857586da'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:53:26.788407 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-9ae34aaf-3bab-4b09-b521-50e609166696 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c45c0a7451d949d89efe0bcd857586da'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:53:26.789031 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-9ae34aaf-3bab-4b09-b521-50e609166696 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c45c0a7451d949d89efe0bcd857586da'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:53:26.789630 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-9ae34aaf-3bab-4b09-b521-50e609166696 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c45c0a7451d949d89efe0bcd857586da'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:53:26.807101 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1219/2462] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Jul 27 10:53:26 2022] DELETE /identity/v3/users/c45c0a7451d949d89efe0bcd857586da => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:53:27.079968 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52010ad-0f23-4fa3-bf69-315bf57db78c admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:53:27.081526 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-e52010ad-0f23-4fa3-bf69-315bf57db78c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:53:27.095497 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52010ad-0f23-4fa3-bf69-315bf57db78c admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:53:27.152221 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a7054b-4078-4486-b009-e4a0623d66a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:53:27.153507 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a7054b-4078-4486-b009-e4a0623d66a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:53:27.153859 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a7054b-4078-4486-b009-e4a0623d66a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:53:27.154074 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a7054b-4078-4486-b009-e4a0623d66a1 admin admin] PATH_INFO: `/v3/projects/8e90721f2b4847be95c19217190b6273` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:53:27.155259 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a7054b-4078-4486-b009-e4a0623d66a1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8e90721f2b4847be95c19217190b6273)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:53:27.157151 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a7054b-4078-4486-b009-e4a0623d66a1 admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:53:27.203905 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-b5a7054b-4078-4486-b009-e4a0623d66a1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8e90721f2b4847be95c19217190b6273', 'request_id': 'req-b5a7054b-4078-4486-b009-e4a0623d66a1'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:53:27.209943 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-b5a7054b-4078-4486-b009-e4a0623d66a1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8e90721f2b4847be95c19217190b6273', 'request_id': 'req-b5a7054b-4078-4486-b009-e4a0623d66a1'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:53:27.216942 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-b5a7054b-4078-4486-b009-e4a0623d66a1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8e90721f2b4847be95c19217190b6273', 'request_id': 'req-b5a7054b-4078-4486-b009-e4a0623d66a1'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:53:27.227066 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-b5a7054b-4078-4486-b009-e4a0623d66a1 admin admin] The token cache is being invalidate because project 8e90721f2b4847be95c19217190b6273 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86827) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 10:53:27.227356 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-b5a7054b-4078-4486-b009-e4a0623d66a1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:53:27.227975 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-b5a7054b-4078-4486-b009-e4a0623d66a1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:53:27.229462 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1244/2463] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Jul 27 10:53:27 2022] DELETE /identity/v3/projects/8e90721f2b4847be95c19217190b6273 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:53:27.235067 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae0a232-62a8-4196-bbea-3fc38c2fb55e None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:53:27.235570 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae0a232-62a8-4196-bbea-3fc38c2fb55e None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:53:27.235722 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae0a232-62a8-4196-bbea-3fc38c2fb55e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:53:27.256376 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-5ae0a232-62a8-4196-bbea-3fc38c2fb55e None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:53:27.257854 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-5ae0a232-62a8-4196-bbea-3fc38c2fb55e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:53:27.321128 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1220/2464] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:53:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:53:27.387505 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ff8d60-4b68-43df-b2d4-8d8f4ce9fd45 None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:53:27.387791 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ff8d60-4b68-43df-b2d4-8d8f4ce9fd45 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:53:27.388102 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ff8d60-4b68-43df-b2d4-8d8f4ce9fd45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:53:27.407535 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-c6ff8d60-4b68-43df-b2d4-8d8f4ce9fd45 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:53:27.408684 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-c6ff8d60-4b68-43df-b2d4-8d8f4ce9fd45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:53:27.416701 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1245/2465] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:53:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:53:27.421558 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae0a232-62a8-4196-bbea-3fc38c2fb55e None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:53:27.434875 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae0a232-62a8-4196-bbea-3fc38c2fb55e None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:53:27.439571 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893957b1-1513-4f42-a70b-f15499138813 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:53:27.440198 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893957b1-1513-4f42-a70b-f15499138813 admin admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:53:27.440597 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893957b1-1513-4f42-a70b-f15499138813 admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:53:27.440878 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893957b1-1513-4f42-a70b-f15499138813 admin admin] PATH_INFO: `/v3/domains` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:53:27.441584 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893957b1-1513-4f42-a70b-f15499138813 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86828) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:53:27.441895 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893957b1-1513-4f42-a70b-f15499138813 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:53:27.443807 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893957b1-1513-4f42-a70b-f15499138813 admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:53:27.457369 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1221/2466] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Jul 27 10:53:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:53:27.461875 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ff8d60-4b68-43df-b2d4-8d8f4ce9fd45 None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:53:27.472263 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ff8d60-4b68-43df-b2d4-8d8f4ce9fd45 None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:53:27.477352 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfa94b3-4d38-4957-b005-a107905415ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:53:27.477963 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfa94b3-4d38-4957-b005-a107905415ef admin admin] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:53:27.478465 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfa94b3-4d38-4957-b005-a107905415ef admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:53:27.478798 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfa94b3-4d38-4957-b005-a107905415ef admin admin] PATH_INFO: `/v3/projects` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:53:27.479477 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfa94b3-4d38-4957-b005-a107905415ef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:53:27.481968 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfa94b3-4d38-4957-b005-a107905415ef admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:53:27.487467 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:53:27.511475 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1246/2467] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Jul 27 10:53:27 2022] POST /identity/v3/projects => generated 381 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 10:53:27.516229 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893957b1-1513-4f42-a70b-f15499138813 admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:53:27.526161 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893957b1-1513-4f42-a70b-f15499138813 admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:53:27.566674 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:53:27.586964 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63e9245-0915-4c0f-97e8-695956fb022c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:53:27.587587 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63e9245-0915-4c0f-97e8-695956fb022c admin admin] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:53:27.587869 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63e9245-0915-4c0f-97e8-695956fb022c admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:53:27.588168 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63e9245-0915-4c0f-97e8-695956fb022c admin admin] PATH_INFO: `/v3/users` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:53:27.588855 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63e9245-0915-4c0f-97e8-695956fb022c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:53:27.590862 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63e9245-0915-4c0f-97e8-695956fb022c admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:53:27.610659 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1222/2468] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Jul 27 10:53:27 2022] POST /identity/v3/users => generated 291 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 10:53:27.614765 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfa94b3-4d38-4957-b005-a107905415ef admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:53:27.625616 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfa94b3-4d38-4957-b005-a107905415ef admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:53:27.630136 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babd08cb-af67-4ce1-b3f5-7cfb3500de56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:53:27.630778 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babd08cb-af67-4ce1-b3f5-7cfb3500de56 admin admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:53:27.631059 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babd08cb-af67-4ce1-b3f5-7cfb3500de56 admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:53:27.631332 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babd08cb-af67-4ce1-b3f5-7cfb3500de56 admin admin] PATH_INFO: `/v3/roles` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:53:27.632048 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-babd08cb-af67-4ce1-b3f5-7cfb3500de56 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:53:27.633622 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-babd08cb-af67-4ce1-b3f5-7cfb3500de56 admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:53:27.644163 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1247/2469] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 10:53:27 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 10:53:27.647994 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63e9245-0915-4c0f-97e8-695956fb022c admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:53:27.657841 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63e9245-0915-4c0f-97e8-695956fb022c admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:53:27.662380 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01606698-2fcc-4302-b10e-2d7a3b08963e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:53:27.663016 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01606698-2fcc-4302-b10e-2d7a3b08963e admin admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:53:27.663301 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01606698-2fcc-4302-b10e-2d7a3b08963e admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:53:27.663575 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01606698-2fcc-4302-b10e-2d7a3b08963e admin admin] PATH_INFO: `/v3/roles` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:53:27.664261 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01606698-2fcc-4302-b10e-2d7a3b08963e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:53:27.665854 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01606698-2fcc-4302-b10e-2d7a3b08963e admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:53:27.676224 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1223/2470] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 10:53:27 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 10:53:27.680138 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babd08cb-af67-4ce1-b3f5-7cfb3500de56 admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:53:27.690711 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babd08cb-af67-4ce1-b3f5-7cfb3500de56 admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:53:27.695335 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7434991-55aa-4cfb-8087-23c95613ef4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:53:27.695988 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7434991-55aa-4cfb-8087-23c95613ef4e admin admin] REQUEST_METHOD: `PUT` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:53:27.696272 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7434991-55aa-4cfb-8087-23c95613ef4e admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:53:27.696568 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7434991-55aa-4cfb-8087-23c95613ef4e admin admin] PATH_INFO: `/v3/projects/1fae35ba5b534fdf8416e55c6b26425d/users/852692bc6b534a69ae0f46eea98d90ea/roles/4c9fc0b3baed441da2126be41cc6b04a` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:53:27.713231 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7434991-55aa-4cfb-8087-23c95613ef4e admin admin] RBAC: Authorizing `identity:create_grant(project_id=1fae35ba5b534fdf8416e55c6b26425d, user_id=852692bc6b534a69ae0f46eea98d90ea, role_id=4c9fc0b3baed441da2126be41cc6b04a)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:53:27.715929 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7434991-55aa-4cfb-8087-23c95613ef4e admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:53:27.728338 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1248/2471] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 10:53:27 2022] PUT /identity/v3/projects/1fae35ba5b534fdf8416e55c6b26425d/users/852692bc6b534a69ae0f46eea98d90ea/roles/4c9fc0b3baed441da2126be41cc6b04a => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 10:53:27.738406 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c9b07e-5906-4ee7-b976-742dcdc68f39 None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:53:27.738740 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c9b07e-5906-4ee7-b976-742dcdc68f39 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:53:27.739026 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c9b07e-5906-4ee7-b976-742dcdc68f39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:53:27.769040 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-59c9b07e-5906-4ee7-b976-742dcdc68f39 None None] MFA Rules not processed for user `852692bc6b534a69ae0f46eea98d90ea`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:53:27.770251 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-59c9b07e-5906-4ee7-b976-742dcdc68f39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:53:27.833372 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1224/2472] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:53:27 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:53:27.837966 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7434991-55aa-4cfb-8087-23c95613ef4e admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:53:27.899211 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7434991-55aa-4cfb-8087-23c95613ef4e admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:53:27.904033 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35addc7b-3996-49dc-8756-6da49b6041c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:53:27.904653 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35addc7b-3996-49dc-8756-6da49b6041c9 admin admin] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:53:27.904941 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35addc7b-3996-49dc-8756-6da49b6041c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:53:27.905271 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35addc7b-3996-49dc-8756-6da49b6041c9 admin admin] PATH_INFO: `/v3/projects` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:53:27.905939 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35addc7b-3996-49dc-8756-6da49b6041c9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:53:27.907943 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35addc7b-3996-49dc-8756-6da49b6041c9 admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:53:27.935997 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1249/2473] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Jul 27 10:53:27 2022] POST /identity/v3/projects => generated 381 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 10:53:27.940252 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c9b07e-5906-4ee7-b976-742dcdc68f39 None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:53:27.951910 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c9b07e-5906-4ee7-b976-742dcdc68f39 None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:53:28.008208 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2800a463-5647-47c2-b10c-812b2d26bfef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:53:28.008830 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2800a463-5647-47c2-b10c-812b2d26bfef admin admin] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:53:28.009114 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2800a463-5647-47c2-b10c-812b2d26bfef admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:53:28.009391 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2800a463-5647-47c2-b10c-812b2d26bfef admin admin] PATH_INFO: `/v3/users` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:53:28.010059 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2800a463-5647-47c2-b10c-812b2d26bfef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:53:28.012004 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2800a463-5647-47c2-b10c-812b2d26bfef admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:53:28.032350 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1225/2474] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Jul 27 10:53:27 2022] POST /identity/v3/users => generated 297 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 10:53:28.036571 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35addc7b-3996-49dc-8756-6da49b6041c9 admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:53:28.047995 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35addc7b-3996-49dc-8756-6da49b6041c9 admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:53:28.052606 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c026c2b5-8ce4-4148-8732-12a6bd3f213f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:53:28.053216 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c026c2b5-8ce4-4148-8732-12a6bd3f213f admin admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:53:28.053499 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c026c2b5-8ce4-4148-8732-12a6bd3f213f admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:53:28.053775 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c026c2b5-8ce4-4148-8732-12a6bd3f213f admin admin] PATH_INFO: `/v3/roles` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:53:28.054469 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c026c2b5-8ce4-4148-8732-12a6bd3f213f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:53:28.056106 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c026c2b5-8ce4-4148-8732-12a6bd3f213f admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:53:28.067072 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1250/2475] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 10:53:28 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 10:53:28.071133 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2800a463-5647-47c2-b10c-812b2d26bfef admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:53:28.080978 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2800a463-5647-47c2-b10c-812b2d26bfef admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:53:28.085679 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0bf4f9-f7cf-4ad7-99d5-9769f7224619 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:53:28.086341 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0bf4f9-f7cf-4ad7-99d5-9769f7224619 admin admin] REQUEST_METHOD: `PUT` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:53:28.086645 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0bf4f9-f7cf-4ad7-99d5-9769f7224619 admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:53:28.086925 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0bf4f9-f7cf-4ad7-99d5-9769f7224619 admin admin] PATH_INFO: `/v3/projects/9f432107ac2f4d71817c62f6d5256ece/users/0a671d74057a486b863e39c73070d4ad/roles/21156560eb6b436bb567a40b4976f3f4` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:53:28.101159 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0bf4f9-f7cf-4ad7-99d5-9769f7224619 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f432107ac2f4d71817c62f6d5256ece, user_id=0a671d74057a486b863e39c73070d4ad, role_id=21156560eb6b436bb567a40b4976f3f4)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:53:28.103816 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0bf4f9-f7cf-4ad7-99d5-9769f7224619 admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:53:28.117022 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1226/2476] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 10:53:28 2022] PUT /identity/v3/projects/9f432107ac2f4d71817c62f6d5256ece/users/0a671d74057a486b863e39c73070d4ad/roles/21156560eb6b436bb567a40b4976f3f4 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:53:28.121080 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c026c2b5-8ce4-4148-8732-12a6bd3f213f admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:53:28.131922 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c026c2b5-8ce4-4148-8732-12a6bd3f213f admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:53:28.188434 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e8cfed-eb6a-41b8-812d-601e38015ff8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:53:28.189032 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e8cfed-eb6a-41b8-812d-601e38015ff8 admin admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:53:28.189250 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e8cfed-eb6a-41b8-812d-601e38015ff8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:53:28.189525 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e8cfed-eb6a-41b8-812d-601e38015ff8 admin admin] PATH_INFO: `/v3/roles` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:53:28.190215 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e8cfed-eb6a-41b8-812d-601e38015ff8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:53:28.191853 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e8cfed-eb6a-41b8-812d-601e38015ff8 admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:53:28.201497 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1251/2477] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 10:53:28 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 10:53:28.205872 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0bf4f9-f7cf-4ad7-99d5-9769f7224619 admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:53:28.216950 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0bf4f9-f7cf-4ad7-99d5-9769f7224619 admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:53:28.221661 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488a77e7-c9aa-477e-a0e0-ed4d0a1994ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:53:28.222324 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488a77e7-c9aa-477e-a0e0-ed4d0a1994ba admin admin] REQUEST_METHOD: `PUT` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:53:28.222634 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488a77e7-c9aa-477e-a0e0-ed4d0a1994ba admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:53:28.222911 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488a77e7-c9aa-477e-a0e0-ed4d0a1994ba admin admin] PATH_INFO: `/v3/projects/9f432107ac2f4d71817c62f6d5256ece/users/0a671d74057a486b863e39c73070d4ad/roles/21156560eb6b436bb567a40b4976f3f4` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:53:28.224884 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488a77e7-c9aa-477e-a0e0-ed4d0a1994ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f432107ac2f4d71817c62f6d5256ece, user_id=0a671d74057a486b863e39c73070d4ad, role_id=21156560eb6b436bb567a40b4976f3f4)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:53:28.227560 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488a77e7-c9aa-477e-a0e0-ed4d0a1994ba admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:53:28.234255 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:53:28.239425 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1227/2478] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 10:53:28 2022] PUT /identity/v3/projects/9f432107ac2f4d71817c62f6d5256ece/users/0a671d74057a486b863e39c73070d4ad/roles/21156560eb6b436bb567a40b4976f3f4 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:53:28.249758 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32a0f72-0c7c-45ad-82b2-02cfaea9361e None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:53:28.250049 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32a0f72-0c7c-45ad-82b2-02cfaea9361e None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:53:28.250327 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32a0f72-0c7c-45ad-82b2-02cfaea9361e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:53:28.278147 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-e32a0f72-0c7c-45ad-82b2-02cfaea9361e None None] MFA Rules not processed for user `0a671d74057a486b863e39c73070d4ad`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:53:28.279387 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-e32a0f72-0c7c-45ad-82b2-02cfaea9361e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:53:28.288443 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:53:28.347091 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1252/2479] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:53:28 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:53:28.360878 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488a77e7-c9aa-477e-a0e0-ed4d0a1994ba admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:53:28.363355 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-488a77e7-c9aa-477e-a0e0-ed4d0a1994ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:53:28.422776 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488a77e7-c9aa-477e-a0e0-ed4d0a1994ba admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:53:28.426415 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36da5dda-d728-4fb0-bb08-51a8f794eb1f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '864d35f268cf45148756109759bb2743', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:53:28.427076 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36da5dda-d728-4fb0-bb08-51a8f794eb1f service cinder] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:53:28.427358 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36da5dda-d728-4fb0-bb08-51a8f794eb1f service cinder] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:53:28.427726 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36da5dda-d728-4fb0-bb08-51a8f794eb1f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:53:28.436601 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36da5dda-d728-4fb0-bb08-51a8f794eb1f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:53:28.438022 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36da5dda-d728-4fb0-bb08-51a8f794eb1f service cinder] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:53:28.447591 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1228/2480] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:53:28 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 10:53:28.767113 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32a0f72-0c7c-45ad-82b2-02cfaea9361e None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:53:28.779551 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32a0f72-0c7c-45ad-82b2-02cfaea9361e None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:53:28.785664 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e5228d-0b8b-45ac-8688-f279a61662a8 tempest-VolumeQuotaClassesTest-1059533273 tempest-VolumeQuotaClassesTest-1059533273-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a671d74057a486b863e39c73070d4ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f432107ac2f4d71817c62f6d5256ece', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:53:28.786456 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e5228d-0b8b-45ac-8688-f279a61662a8 tempest-VolumeQuotaClassesTest-1059533273 tempest-VolumeQuotaClassesTest-1059533273-project-admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:53:28.786844 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e5228d-0b8b-45ac-8688-f279a61662a8 tempest-VolumeQuotaClassesTest-1059533273 tempest-VolumeQuotaClassesTest-1059533273-project-admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:53:28.787203 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e5228d-0b8b-45ac-8688-f279a61662a8 tempest-VolumeQuotaClassesTest-1059533273 tempest-VolumeQuotaClassesTest-1059533273-project-admin] PATH_INFO: `/v3/domains` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:53:28.788092 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e5228d-0b8b-45ac-8688-f279a61662a8 tempest-VolumeQuotaClassesTest-1059533273 tempest-VolumeQuotaClassesTest-1059533273-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=86827) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:53:28.788486 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e5228d-0b8b-45ac-8688-f279a61662a8 tempest-VolumeQuotaClassesTest-1059533273 tempest-VolumeQuotaClassesTest-1059533273-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:53:28.790238 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e5228d-0b8b-45ac-8688-f279a61662a8 tempest-VolumeQuotaClassesTest-1059533273 tempest-VolumeQuotaClassesTest-1059533273-project-admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:53:28.803187 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1253/2481] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Jul 27 10:53:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:53:28.807375 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36da5dda-d728-4fb0-bb08-51a8f794eb1f service cinder] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:53:28.816500 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36da5dda-d728-4fb0-bb08-51a8f794eb1f service cinder] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:53:28.822033 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44024169-361b-4428-b66b-ac22fab4fadc tempest-VolumeQuotaClassesTest-1059533273 tempest-VolumeQuotaClassesTest-1059533273-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a671d74057a486b863e39c73070d4ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f432107ac2f4d71817c62f6d5256ece', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:53:28.822981 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44024169-361b-4428-b66b-ac22fab4fadc tempest-VolumeQuotaClassesTest-1059533273 tempest-VolumeQuotaClassesTest-1059533273-project-admin] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:53:28.823273 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44024169-361b-4428-b66b-ac22fab4fadc tempest-VolumeQuotaClassesTest-1059533273 tempest-VolumeQuotaClassesTest-1059533273-project-admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:53:28.823597 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44024169-361b-4428-b66b-ac22fab4fadc tempest-VolumeQuotaClassesTest-1059533273 tempest-VolumeQuotaClassesTest-1059533273-project-admin] PATH_INFO: `/v3/projects` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:53:28.824755 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44024169-361b-4428-b66b-ac22fab4fadc tempest-VolumeQuotaClassesTest-1059533273 tempest-VolumeQuotaClassesTest-1059533273-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:53:28.826697 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44024169-361b-4428-b66b-ac22fab4fadc tempest-VolumeQuotaClassesTest-1059533273 tempest-VolumeQuotaClassesTest-1059533273-project-admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:53:28.856015 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1229/2482] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Jul 27 10:53:28 2022] POST /identity/v3/projects => generated 354 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 10:53:28.862304 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e5228d-0b8b-45ac-8688-f279a61662a8 tempest-VolumeQuotaClassesTest-1059533273 tempest-VolumeQuotaClassesTest-1059533273-project-admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:53:28.872953 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e5228d-0b8b-45ac-8688-f279a61662a8 tempest-VolumeQuotaClassesTest-1059533273 tempest-VolumeQuotaClassesTest-1059533273-project-admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:53:28.930886 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ec2292-ef19-4c98-9a53-5601b85d3d17 tempest-VolumeQuotaClassesTest-1059533273 tempest-VolumeQuotaClassesTest-1059533273-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a671d74057a486b863e39c73070d4ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f432107ac2f4d71817c62f6d5256ece', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:53:28.931516 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ec2292-ef19-4c98-9a53-5601b85d3d17 tempest-VolumeQuotaClassesTest-1059533273 tempest-VolumeQuotaClassesTest-1059533273-project-admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:53:28.931813 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ec2292-ef19-4c98-9a53-5601b85d3d17 tempest-VolumeQuotaClassesTest-1059533273 tempest-VolumeQuotaClassesTest-1059533273-project-admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:53:28.932095 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ec2292-ef19-4c98-9a53-5601b85d3d17 tempest-VolumeQuotaClassesTest-1059533273 tempest-VolumeQuotaClassesTest-1059533273-project-admin] PATH_INFO: `/v3/domains` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:53:28.932801 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ec2292-ef19-4c98-9a53-5601b85d3d17 tempest-VolumeQuotaClassesTest-1059533273 tempest-VolumeQuotaClassesTest-1059533273-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=86827) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:53:28.933112 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ec2292-ef19-4c98-9a53-5601b85d3d17 tempest-VolumeQuotaClassesTest-1059533273 tempest-VolumeQuotaClassesTest-1059533273-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:53:28.934742 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ec2292-ef19-4c98-9a53-5601b85d3d17 tempest-VolumeQuotaClassesTest-1059533273 tempest-VolumeQuotaClassesTest-1059533273-project-admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:53:28.954798 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1254/2483] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Jul 27 10:53:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:53:28.988222 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44024169-361b-4428-b66b-ac22fab4fadc tempest-VolumeQuotaClassesTest-1059533273 tempest-VolumeQuotaClassesTest-1059533273-project-admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:53:28.999512 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44024169-361b-4428-b66b-ac22fab4fadc tempest-VolumeQuotaClassesTest-1059533273 tempest-VolumeQuotaClassesTest-1059533273-project-admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:53:29.004151 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb9c71e-d698-41f3-8d7d-b194d2e22369 tempest-VolumeQuotaClassesTest-1059533273 tempest-VolumeQuotaClassesTest-1059533273-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a671d74057a486b863e39c73070d4ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f432107ac2f4d71817c62f6d5256ece', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:53:29.004859 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb9c71e-d698-41f3-8d7d-b194d2e22369 tempest-VolumeQuotaClassesTest-1059533273 tempest-VolumeQuotaClassesTest-1059533273-project-admin] REQUEST_METHOD: `DELETE` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:53:29.005271 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb9c71e-d698-41f3-8d7d-b194d2e22369 tempest-VolumeQuotaClassesTest-1059533273 tempest-VolumeQuotaClassesTest-1059533273-project-admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:53:29.005555 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb9c71e-d698-41f3-8d7d-b194d2e22369 tempest-VolumeQuotaClassesTest-1059533273 tempest-VolumeQuotaClassesTest-1059533273-project-admin] PATH_INFO: `/v3/projects/f01ee49c45904339862e4e3a3bc1dea4` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:53:29.006774 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb9c71e-d698-41f3-8d7d-b194d2e22369 tempest-VolumeQuotaClassesTest-1059533273 tempest-VolumeQuotaClassesTest-1059533273-project-admin] RBAC: Authorizing `identity:delete_project(project_id=f01ee49c45904339862e4e3a3bc1dea4)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:53:29.008672 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb9c71e-d698-41f3-8d7d-b194d2e22369 tempest-VolumeQuotaClassesTest-1059533273 tempest-VolumeQuotaClassesTest-1059533273-project-admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:53:29.058546 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-beb9c71e-d698-41f3-8d7d-b194d2e22369 tempest-VolumeQuotaClassesTest-1059533273 tempest-VolumeQuotaClassesTest-1059533273-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f01ee49c45904339862e4e3a3bc1dea4', 'request_id': 'req-beb9c71e-d698-41f3-8d7d-b194d2e22369'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:53:29.065980 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-beb9c71e-d698-41f3-8d7d-b194d2e22369 tempest-VolumeQuotaClassesTest-1059533273 tempest-VolumeQuotaClassesTest-1059533273-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f01ee49c45904339862e4e3a3bc1dea4', 'request_id': 'req-beb9c71e-d698-41f3-8d7d-b194d2e22369'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:53:29.073737 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-beb9c71e-d698-41f3-8d7d-b194d2e22369 tempest-VolumeQuotaClassesTest-1059533273 tempest-VolumeQuotaClassesTest-1059533273-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f01ee49c45904339862e4e3a3bc1dea4', 'request_id': 'req-beb9c71e-d698-41f3-8d7d-b194d2e22369'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:53:29.085097 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-beb9c71e-d698-41f3-8d7d-b194d2e22369 tempest-VolumeQuotaClassesTest-1059533273 tempest-VolumeQuotaClassesTest-1059533273-project-admin] The token cache is being invalidate because project f01ee49c45904339862e4e3a3bc1dea4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86828) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 10:53:29.085405 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-beb9c71e-d698-41f3-8d7d-b194d2e22369 tempest-VolumeQuotaClassesTest-1059533273 tempest-VolumeQuotaClassesTest-1059533273-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:53:29.086053 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-beb9c71e-d698-41f3-8d7d-b194d2e22369 tempest-VolumeQuotaClassesTest-1059533273 tempest-VolumeQuotaClassesTest-1059533273-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:53:29.088306 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1230/2484] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Jul 27 10:53:28 2022] DELETE /identity/v3/projects/f01ee49c45904339862e4e3a3bc1dea4 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:53:29.306890 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ec2292-ef19-4c98-9a53-5601b85d3d17 tempest-VolumeQuotaClassesTest-1059533273 tempest-VolumeQuotaClassesTest-1059533273-project-admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:53:29.309287 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-10ec2292-ef19-4c98-9a53-5601b85d3d17 tempest-VolumeQuotaClassesTest-1059533273 tempest-VolumeQuotaClassesTest-1059533273-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:53:29.377865 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ec2292-ef19-4c98-9a53-5601b85d3d17 tempest-VolumeQuotaClassesTest-1059533273 tempest-VolumeQuotaClassesTest-1059533273-project-admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:53:29.408785 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:53:29.433285 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848472ef-6366-4b50-9a23-0d22c315ae8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:53:29.434572 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848472ef-6366-4b50-9a23-0d22c315ae8b admin admin] REQUEST_METHOD: `DELETE` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:53:29.434888 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848472ef-6366-4b50-9a23-0d22c315ae8b admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:53:29.435167 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848472ef-6366-4b50-9a23-0d22c315ae8b admin admin] PATH_INFO: `/v3/users/852692bc6b534a69ae0f46eea98d90ea` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:53:29.436334 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848472ef-6366-4b50-9a23-0d22c315ae8b admin admin] RBAC: Authorizing `identity:delete_user(user_id=852692bc6b534a69ae0f46eea98d90ea)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:53:29.438249 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848472ef-6366-4b50-9a23-0d22c315ae8b admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:53:29.482218 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-848472ef-6366-4b50-9a23-0d22c315ae8b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '852692bc6b534a69ae0f46eea98d90ea'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:53:29.488938 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-848472ef-6366-4b50-9a23-0d22c315ae8b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '852692bc6b534a69ae0f46eea98d90ea'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:53:29.496020 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-848472ef-6366-4b50-9a23-0d22c315ae8b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '852692bc6b534a69ae0f46eea98d90ea'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:53:29.497169 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-848472ef-6366-4b50-9a23-0d22c315ae8b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '852692bc6b534a69ae0f46eea98d90ea'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:53:29.497825 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-848472ef-6366-4b50-9a23-0d22c315ae8b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '852692bc6b534a69ae0f46eea98d90ea'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:53:29.515352 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1255/2485] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Jul 27 10:53:29 2022] DELETE /identity/v3/users/852692bc6b534a69ae0f46eea98d90ea => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:53:29.519904 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d94f83-1461-45cf-9c30-e8902583c7fc None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:53:29.520470 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d94f83-1461-45cf-9c30-e8902583c7fc None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:53:29.520470 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d94f83-1461-45cf-9c30-e8902583c7fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:53:29.539718 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-a5d94f83-1461-45cf-9c30-e8902583c7fc None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:53:29.540916 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-a5d94f83-1461-45cf-9c30-e8902583c7fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:53:29.598189 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1231/2486] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:53:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 10:53:29.608499 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848472ef-6366-4b50-9a23-0d22c315ae8b admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:53:29.609884 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-848472ef-6366-4b50-9a23-0d22c315ae8b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:53:29.665996 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848472ef-6366-4b50-9a23-0d22c315ae8b admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:53:29.670151 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820307c0-c5e9-44e4-b4ab-b14d972846de service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7922d0a1a7c4e808d0b2a80dca0da40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:53:29.670803 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820307c0-c5e9-44e4-b4ab-b14d972846de service neutron] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:53:29.671082 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820307c0-c5e9-44e4-b4ab-b14d972846de service neutron] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:53:29.671351 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820307c0-c5e9-44e4-b4ab-b14d972846de service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:53:29.680221 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820307c0-c5e9-44e4-b4ab-b14d972846de service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:53:29.681648 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820307c0-c5e9-44e4-b4ab-b14d972846de service neutron] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:53:29.691600 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1256/2487] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:53:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 10:53:29.956156 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d94f83-1461-45cf-9c30-e8902583c7fc None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:53:29.957586 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-a5d94f83-1461-45cf-9c30-e8902583c7fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:53:29.967732 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:53:29.972294 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d94f83-1461-45cf-9c30-e8902583c7fc None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:53:29.977727 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a346c5a2-e525-4ed1-8fa4-9c60c178fcd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:53:29.978693 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a346c5a2-e525-4ed1-8fa4-9c60c178fcd1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:53:29.979062 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a346c5a2-e525-4ed1-8fa4-9c60c178fcd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:53:29.979573 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a346c5a2-e525-4ed1-8fa4-9c60c178fcd1 admin admin] PATH_INFO: `/v3/projects/1fae35ba5b534fdf8416e55c6b26425d` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:53:29.981737 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a346c5a2-e525-4ed1-8fa4-9c60c178fcd1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1fae35ba5b534fdf8416e55c6b26425d)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:53:29.983971 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a346c5a2-e525-4ed1-8fa4-9c60c178fcd1 admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:53:30.032521 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-a346c5a2-e525-4ed1-8fa4-9c60c178fcd1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1fae35ba5b534fdf8416e55c6b26425d', 'request_id': 'req-a346c5a2-e525-4ed1-8fa4-9c60c178fcd1'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:53:30.038759 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-a346c5a2-e525-4ed1-8fa4-9c60c178fcd1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1fae35ba5b534fdf8416e55c6b26425d', 'request_id': 'req-a346c5a2-e525-4ed1-8fa4-9c60c178fcd1'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:53:30.046555 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-a346c5a2-e525-4ed1-8fa4-9c60c178fcd1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1fae35ba5b534fdf8416e55c6b26425d', 'request_id': 'req-a346c5a2-e525-4ed1-8fa4-9c60c178fcd1'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:53:30.056970 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-a346c5a2-e525-4ed1-8fa4-9c60c178fcd1 admin admin] The token cache is being invalidate because project 1fae35ba5b534fdf8416e55c6b26425d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86828) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 10:53:30.057260 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-a346c5a2-e525-4ed1-8fa4-9c60c178fcd1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:53:30.057886 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-a346c5a2-e525-4ed1-8fa4-9c60c178fcd1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:53:30.059471 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1232/2488] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Jul 27 10:53:29 2022] DELETE /identity/v3/projects/1fae35ba5b534fdf8416e55c6b26425d => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:53:30.063468 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820307c0-c5e9-44e4-b4ab-b14d972846de service neutron] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:53:30.073073 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820307c0-c5e9-44e4-b4ab-b14d972846de service neutron] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:53:30.127270 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:53:30.134695 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491525a8-293b-4270-a090-225bf5257f7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:53:30.135417 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491525a8-293b-4270-a090-225bf5257f7e admin admin] REQUEST_METHOD: `DELETE` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:53:30.135898 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491525a8-293b-4270-a090-225bf5257f7e admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:53:30.136340 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491525a8-293b-4270-a090-225bf5257f7e admin admin] PATH_INFO: `/v3/users/0a671d74057a486b863e39c73070d4ad` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:53:30.138043 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491525a8-293b-4270-a090-225bf5257f7e admin admin] RBAC: Authorizing `identity:delete_user(user_id=0a671d74057a486b863e39c73070d4ad)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:53:30.140560 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491525a8-293b-4270-a090-225bf5257f7e admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:53:30.186352 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-491525a8-293b-4270-a090-225bf5257f7e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0a671d74057a486b863e39c73070d4ad'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:53:30.193328 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-491525a8-293b-4270-a090-225bf5257f7e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0a671d74057a486b863e39c73070d4ad'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:53:30.200292 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-491525a8-293b-4270-a090-225bf5257f7e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0a671d74057a486b863e39c73070d4ad'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:53:30.200919 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-491525a8-293b-4270-a090-225bf5257f7e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0a671d74057a486b863e39c73070d4ad'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:53:30.201575 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-491525a8-293b-4270-a090-225bf5257f7e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0a671d74057a486b863e39c73070d4ad'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:53:30.218710 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1257/2489] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Jul 27 10:53:30 2022] DELETE /identity/v3/users/0a671d74057a486b863e39c73070d4ad => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:53:30.501360 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a346c5a2-e525-4ed1-8fa4-9c60c178fcd1 admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:53:30.503613 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-a346c5a2-e525-4ed1-8fa4-9c60c178fcd1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:53:30.518688 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a346c5a2-e525-4ed1-8fa4-9c60c178fcd1 admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:53:30.588798 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d40c9a2-d3c2-42b7-b6d5-1444580766b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:53:30.589468 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d40c9a2-d3c2-42b7-b6d5-1444580766b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:53:30.589748 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d40c9a2-d3c2-42b7-b6d5-1444580766b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:53:30.590048 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d40c9a2-d3c2-42b7-b6d5-1444580766b1 admin admin] PATH_INFO: `/v3/projects/9f432107ac2f4d71817c62f6d5256ece` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:53:30.591241 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d40c9a2-d3c2-42b7-b6d5-1444580766b1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9f432107ac2f4d71817c62f6d5256ece)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:53:30.593264 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d40c9a2-d3c2-42b7-b6d5-1444580766b1 admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:53:30.642848 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-0d40c9a2-d3c2-42b7-b6d5-1444580766b1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9f432107ac2f4d71817c62f6d5256ece', 'request_id': 'req-0d40c9a2-d3c2-42b7-b6d5-1444580766b1'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:53:30.655065 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-0d40c9a2-d3c2-42b7-b6d5-1444580766b1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9f432107ac2f4d71817c62f6d5256ece', 'request_id': 'req-0d40c9a2-d3c2-42b7-b6d5-1444580766b1'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:53:30.663393 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-0d40c9a2-d3c2-42b7-b6d5-1444580766b1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9f432107ac2f4d71817c62f6d5256ece', 'request_id': 'req-0d40c9a2-d3c2-42b7-b6d5-1444580766b1'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:53:30.675426 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-0d40c9a2-d3c2-42b7-b6d5-1444580766b1 admin admin] The token cache is being invalidate because project 9f432107ac2f4d71817c62f6d5256ece was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86828) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 10:53:30.675785 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-0d40c9a2-d3c2-42b7-b6d5-1444580766b1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:53:30.676370 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-0d40c9a2-d3c2-42b7-b6d5-1444580766b1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:53:30.678309 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1233/2490] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Jul 27 10:53:30 2022] DELETE /identity/v3/projects/9f432107ac2f4d71817c62f6d5256ece => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:53:30.683096 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b7b527-8d4b-489f-af53-10e916ade076 None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:53:30.684513 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b7b527-8d4b-489f-af53-10e916ade076 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:53:30.684735 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b7b527-8d4b-489f-af53-10e916ade076 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:53:30.711732 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-d9b7b527-8d4b-489f-af53-10e916ade076 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:53:30.712960 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-d9b7b527-8d4b-489f-af53-10e916ade076 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:53:30.774989 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1258/2491] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:53:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:53:30.782663 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4570daec-fcbb-4481-bcd6-87ae78b913c8 None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:53:30.782956 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4570daec-fcbb-4481-bcd6-87ae78b913c8 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:53:30.783235 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4570daec-fcbb-4481-bcd6-87ae78b913c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:53:30.804514 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-4570daec-fcbb-4481-bcd6-87ae78b913c8 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:53:30.805789 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-4570daec-fcbb-4481-bcd6-87ae78b913c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:53:30.814643 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1234/2492] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:53:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:53:30.819213 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b7b527-8d4b-489f-af53-10e916ade076 None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:53:30.829250 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b7b527-8d4b-489f-af53-10e916ade076 None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:53:30.833748 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9a9e2f-bd1d-424f-bca2-aa541e342388 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:53:30.834352 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9a9e2f-bd1d-424f-bca2-aa541e342388 admin admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:53:30.834655 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9a9e2f-bd1d-424f-bca2-aa541e342388 admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:53:30.834935 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9a9e2f-bd1d-424f-bca2-aa541e342388 admin admin] PATH_INFO: `/v3/domains` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:53:30.835622 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9a9e2f-bd1d-424f-bca2-aa541e342388 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86827) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:53:30.835925 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9a9e2f-bd1d-424f-bca2-aa541e342388 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:53:30.837511 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9a9e2f-bd1d-424f-bca2-aa541e342388 admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:53:30.848817 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1259/2493] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Jul 27 10:53:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:53:30.852994 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4570daec-fcbb-4481-bcd6-87ae78b913c8 None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:53:30.862675 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4570daec-fcbb-4481-bcd6-87ae78b913c8 None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:53:30.867379 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624bfbe4-665e-4a5c-bbbd-b6decff7eb33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:53:30.867983 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624bfbe4-665e-4a5c-bbbd-b6decff7eb33 admin admin] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:53:30.868263 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624bfbe4-665e-4a5c-bbbd-b6decff7eb33 admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:53:30.868538 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624bfbe4-665e-4a5c-bbbd-b6decff7eb33 admin admin] PATH_INFO: `/v3/projects` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:53:30.869194 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624bfbe4-665e-4a5c-bbbd-b6decff7eb33 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:53:30.871185 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624bfbe4-665e-4a5c-bbbd-b6decff7eb33 admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:53:30.881200 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:53:30.902908 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1235/2494] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Jul 27 10:53:30 2022] POST /identity/v3/projects => generated 365 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 10:53:30.907206 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9a9e2f-bd1d-424f-bca2-aa541e342388 admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:53:30.917618 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9a9e2f-bd1d-424f-bca2-aa541e342388 admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:53:30.974838 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b41c5e5-466d-4194-8b0a-ba5dec5442e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:53:30.975605 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b41c5e5-466d-4194-8b0a-ba5dec5442e1 admin admin] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:53:30.975914 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b41c5e5-466d-4194-8b0a-ba5dec5442e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:53:30.976211 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b41c5e5-466d-4194-8b0a-ba5dec5442e1 admin admin] PATH_INFO: `/v3/users` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:53:30.976911 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b41c5e5-466d-4194-8b0a-ba5dec5442e1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:53:30.979061 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b41c5e5-466d-4194-8b0a-ba5dec5442e1 admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:53:30.989204 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:53:31.002209 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1260/2495] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Jul 27 10:53:30 2022] POST /identity/v3/users => generated 283 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 10:53:31.006508 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624bfbe4-665e-4a5c-bbbd-b6decff7eb33 admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:53:31.017139 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624bfbe4-665e-4a5c-bbbd-b6decff7eb33 admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:53:31.022140 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36ffb1b-6b76-4b0f-b13b-fd55fd68a88d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:53:31.022737 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36ffb1b-6b76-4b0f-b13b-fd55fd68a88d admin admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:53:31.023076 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36ffb1b-6b76-4b0f-b13b-fd55fd68a88d admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:53:31.023356 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36ffb1b-6b76-4b0f-b13b-fd55fd68a88d admin admin] PATH_INFO: `/v3/roles` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:53:31.024109 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36ffb1b-6b76-4b0f-b13b-fd55fd68a88d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:53:31.025764 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36ffb1b-6b76-4b0f-b13b-fd55fd68a88d admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:53:31.036592 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1236/2496] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 10:53:31 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 10:53:31.040648 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b41c5e5-466d-4194-8b0a-ba5dec5442e1 admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:53:31.051578 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b41c5e5-466d-4194-8b0a-ba5dec5442e1 admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:53:31.057433 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d1c5d8-e88f-4650-99a8-d16838c78be1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:53:31.058006 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d1c5d8-e88f-4650-99a8-d16838c78be1 admin admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:53:31.058288 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d1c5d8-e88f-4650-99a8-d16838c78be1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:53:31.058567 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d1c5d8-e88f-4650-99a8-d16838c78be1 admin admin] PATH_INFO: `/v3/roles` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:53:31.059327 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d1c5d8-e88f-4650-99a8-d16838c78be1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:53:31.060925 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d1c5d8-e88f-4650-99a8-d16838c78be1 admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:53:31.073931 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1261/2497] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 10:53:31 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 10:53:31.077211 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36ffb1b-6b76-4b0f-b13b-fd55fd68a88d admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:53:31.087198 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36ffb1b-6b76-4b0f-b13b-fd55fd68a88d admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:53:31.092382 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed72901-5dff-466d-bd88-ca131680337a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:53:31.093038 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed72901-5dff-466d-bd88-ca131680337a admin admin] REQUEST_METHOD: `PUT` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:53:31.093320 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed72901-5dff-466d-bd88-ca131680337a admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:53:31.093597 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed72901-5dff-466d-bd88-ca131680337a admin admin] PATH_INFO: `/v3/projects/7695445cad964faeaa78f760628fdf5a/users/8baae9fe605a48efa95c8be7437982a7/roles/4c9fc0b3baed441da2126be41cc6b04a` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:53:31.107840 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed72901-5dff-466d-bd88-ca131680337a admin admin] RBAC: Authorizing `identity:create_grant(project_id=7695445cad964faeaa78f760628fdf5a, user_id=8baae9fe605a48efa95c8be7437982a7, role_id=4c9fc0b3baed441da2126be41cc6b04a)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:53:31.110490 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed72901-5dff-466d-bd88-ca131680337a admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:53:31.122968 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1237/2498] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 10:53:31 2022] PUT /identity/v3/projects/7695445cad964faeaa78f760628fdf5a/users/8baae9fe605a48efa95c8be7437982a7/roles/4c9fc0b3baed441da2126be41cc6b04a => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:53:31.133355 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e222a9bc-c2b4-4b69-8ff7-e937bc5a6509 None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:53:31.133648 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e222a9bc-c2b4-4b69-8ff7-e937bc5a6509 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:53:31.133927 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e222a9bc-c2b4-4b69-8ff7-e937bc5a6509 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:53:31.161664 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-e222a9bc-c2b4-4b69-8ff7-e937bc5a6509 None None] MFA Rules not processed for user `8baae9fe605a48efa95c8be7437982a7`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:53:31.162892 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-e222a9bc-c2b4-4b69-8ff7-e937bc5a6509 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:53:31.226836 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1262/2499] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:53:31 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:53:31.231323 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed72901-5dff-466d-bd88-ca131680337a admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:53:31.292406 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed72901-5dff-466d-bd88-ca131680337a admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:53:31.297095 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60df234b-9144-467a-908b-544bebc9dc09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:53:31.297714 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60df234b-9144-467a-908b-544bebc9dc09 admin admin] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:53:31.297998 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60df234b-9144-467a-908b-544bebc9dc09 admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:53:31.298278 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60df234b-9144-467a-908b-544bebc9dc09 admin admin] PATH_INFO: `/v3/projects` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:53:31.298982 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60df234b-9144-467a-908b-544bebc9dc09 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:53:31.300887 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60df234b-9144-467a-908b-544bebc9dc09 admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:53:31.327956 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1238/2500] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Jul 27 10:53:31 2022] POST /identity/v3/projects => generated 367 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 10:53:31.332067 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e222a9bc-c2b4-4b69-8ff7-e937bc5a6509 None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:53:31.342889 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e222a9bc-c2b4-4b69-8ff7-e937bc5a6509 None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:53:31.398154 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e72dcd2-66d6-4559-bf7e-a93ea21b0edb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:53:31.398814 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e72dcd2-66d6-4559-bf7e-a93ea21b0edb admin admin] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:53:31.399101 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e72dcd2-66d6-4559-bf7e-a93ea21b0edb admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:53:31.399378 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e72dcd2-66d6-4559-bf7e-a93ea21b0edb admin admin] PATH_INFO: `/v3/users` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:53:31.400046 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e72dcd2-66d6-4559-bf7e-a93ea21b0edb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:53:31.402058 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e72dcd2-66d6-4559-bf7e-a93ea21b0edb admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:53:31.423558 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1263/2501] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Jul 27 10:53:31 2022] POST /identity/v3/users => generated 290 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 10:53:31.427546 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60df234b-9144-467a-908b-544bebc9dc09 admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:53:31.438375 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60df234b-9144-467a-908b-544bebc9dc09 admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:53:31.443130 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6105dc-fb8e-4ed5-bd7f-e2d3db132af9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:53:31.443698 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6105dc-fb8e-4ed5-bd7f-e2d3db132af9 admin admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:53:31.443983 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6105dc-fb8e-4ed5-bd7f-e2d3db132af9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:53:31.444259 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6105dc-fb8e-4ed5-bd7f-e2d3db132af9 admin admin] PATH_INFO: `/v3/roles` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:53:31.444946 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6105dc-fb8e-4ed5-bd7f-e2d3db132af9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:53:31.446552 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6105dc-fb8e-4ed5-bd7f-e2d3db132af9 admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:53:31.457956 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1239/2502] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 10:53:31 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 10:53:31.462008 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e72dcd2-66d6-4559-bf7e-a93ea21b0edb admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:53:31.471848 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e72dcd2-66d6-4559-bf7e-a93ea21b0edb admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:53:31.476892 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4178b8ea-1827-4988-a050-5d253c6be85d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:53:31.477685 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4178b8ea-1827-4988-a050-5d253c6be85d admin admin] REQUEST_METHOD: `PUT` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:53:31.477895 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4178b8ea-1827-4988-a050-5d253c6be85d admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:53:31.478174 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4178b8ea-1827-4988-a050-5d253c6be85d admin admin] PATH_INFO: `/v3/projects/0c7b05f0a8f64d31bd84b6277dd79e56/users/1b882ffee1ca4cbfbf5c4310d6cc4fa5/roles/21156560eb6b436bb567a40b4976f3f4` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:53:31.494027 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4178b8ea-1827-4988-a050-5d253c6be85d admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c7b05f0a8f64d31bd84b6277dd79e56, user_id=1b882ffee1ca4cbfbf5c4310d6cc4fa5, role_id=21156560eb6b436bb567a40b4976f3f4)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:53:31.497194 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4178b8ea-1827-4988-a050-5d253c6be85d admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:53:31.509642 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1264/2503] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 10:53:31 2022] PUT /identity/v3/projects/0c7b05f0a8f64d31bd84b6277dd79e56/users/1b882ffee1ca4cbfbf5c4310d6cc4fa5/roles/21156560eb6b436bb567a40b4976f3f4 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:53:31.514302 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6105dc-fb8e-4ed5-bd7f-e2d3db132af9 admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:53:31.524942 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6105dc-fb8e-4ed5-bd7f-e2d3db132af9 admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:53:31.582579 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdb4801-2c76-4c43-b3d7-095d1cc3b475 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:53:31.583178 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdb4801-2c76-4c43-b3d7-095d1cc3b475 admin admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:53:31.583498 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdb4801-2c76-4c43-b3d7-095d1cc3b475 admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:53:31.583834 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdb4801-2c76-4c43-b3d7-095d1cc3b475 admin admin] PATH_INFO: `/v3/roles` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:53:31.584595 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccdb4801-2c76-4c43-b3d7-095d1cc3b475 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:53:31.586287 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccdb4801-2c76-4c43-b3d7-095d1cc3b475 admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:53:31.596072 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1240/2504] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 10:53:31 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 10:53:31.600395 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4178b8ea-1827-4988-a050-5d253c6be85d admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:53:31.611254 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4178b8ea-1827-4988-a050-5d253c6be85d admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:53:31.615885 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe0c73c-88be-408c-adb3-f02b2febd1a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:53:31.616542 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe0c73c-88be-408c-adb3-f02b2febd1a3 admin admin] REQUEST_METHOD: `PUT` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:53:31.616827 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe0c73c-88be-408c-adb3-f02b2febd1a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:53:31.617102 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe0c73c-88be-408c-adb3-f02b2febd1a3 admin admin] PATH_INFO: `/v3/projects/0c7b05f0a8f64d31bd84b6277dd79e56/users/1b882ffee1ca4cbfbf5c4310d6cc4fa5/roles/21156560eb6b436bb567a40b4976f3f4` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:53:31.619084 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe0c73c-88be-408c-adb3-f02b2febd1a3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c7b05f0a8f64d31bd84b6277dd79e56, user_id=1b882ffee1ca4cbfbf5c4310d6cc4fa5, role_id=21156560eb6b436bb567a40b4976f3f4)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:53:31.622350 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe0c73c-88be-408c-adb3-f02b2febd1a3 admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:53:31.633183 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1265/2505] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 10:53:31 2022] PUT /identity/v3/projects/0c7b05f0a8f64d31bd84b6277dd79e56/users/1b882ffee1ca4cbfbf5c4310d6cc4fa5/roles/21156560eb6b436bb567a40b4976f3f4 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:53:31.644463 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a762a0-477b-4065-ae1d-6ea0eb406eaf None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:53:31.644804 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a762a0-477b-4065-ae1d-6ea0eb406eaf None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:53:31.645113 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a762a0-477b-4065-ae1d-6ea0eb406eaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:53:31.673005 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-c0a762a0-477b-4065-ae1d-6ea0eb406eaf None None] MFA Rules not processed for user `1b882ffee1ca4cbfbf5c4310d6cc4fa5`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:53:31.674221 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-c0a762a0-477b-4065-ae1d-6ea0eb406eaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:53:31.686882 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:53:31.749599 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1241/2506] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:53:31 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:53:31.756648 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe0c73c-88be-408c-adb3-f02b2febd1a3 admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:53:31.758435 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-bbe0c73c-88be-408c-adb3-f02b2febd1a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:53:31.786054 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:53:31.816982 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe0c73c-88be-408c-adb3-f02b2febd1a3 admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:53:31.820630 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f8ba33-ad84-4f79-aa0f-69fa89e522aa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '864d35f268cf45148756109759bb2743', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:53:31.821243 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f8ba33-ad84-4f79-aa0f-69fa89e522aa service cinder] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:53:31.821528 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f8ba33-ad84-4f79-aa0f-69fa89e522aa service cinder] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:53:31.821806 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f8ba33-ad84-4f79-aa0f-69fa89e522aa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:53:31.830941 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f8ba33-ad84-4f79-aa0f-69fa89e522aa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:53:31.832375 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f8ba33-ad84-4f79-aa0f-69fa89e522aa service cinder] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:53:31.842001 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1266/2507] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:53:31 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 10:53:31.923621 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a762a0-477b-4065-ae1d-6ea0eb406eaf None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:53:31.933366 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a762a0-477b-4065-ae1d-6ea0eb406eaf None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:53:31.937771 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad11e44e-a62f-453e-b026-6f604b055aea service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '864d35f268cf45148756109759bb2743', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:53:31.938393 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad11e44e-a62f-453e-b026-6f604b055aea service cinder] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:53:31.938712 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad11e44e-a62f-453e-b026-6f604b055aea service cinder] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:53:31.939013 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad11e44e-a62f-453e-b026-6f604b055aea service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:53:32.000924 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad11e44e-a62f-453e-b026-6f604b055aea service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:53:32.002362 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad11e44e-a62f-453e-b026-6f604b055aea service cinder] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:53:32.011241 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1242/2508] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:53:31 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 10:53:36.771297 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f8ba33-ad84-4f79-aa0f-69fa89e522aa service cinder] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:53:36.832449 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f8ba33-ad84-4f79-aa0f-69fa89e522aa service cinder] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:53:36.837061 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d64787-fc1b-4925-92f5-e5be460bfbe2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:53:36.837765 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d64787-fc1b-4925-92f5-e5be460bfbe2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:53:36.838083 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d64787-fc1b-4925-92f5-e5be460bfbe2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:53:36.838393 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d64787-fc1b-4925-92f5-e5be460bfbe2 admin admin] PATH_INFO: `/v3/users/8baae9fe605a48efa95c8be7437982a7` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:53:36.839650 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d64787-fc1b-4925-92f5-e5be460bfbe2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8baae9fe605a48efa95c8be7437982a7)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:53:36.841561 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d64787-fc1b-4925-92f5-e5be460bfbe2 admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:53:36.886656 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-b8d64787-fc1b-4925-92f5-e5be460bfbe2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8baae9fe605a48efa95c8be7437982a7'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:53:36.893687 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-b8d64787-fc1b-4925-92f5-e5be460bfbe2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8baae9fe605a48efa95c8be7437982a7'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:53:36.900493 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-b8d64787-fc1b-4925-92f5-e5be460bfbe2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8baae9fe605a48efa95c8be7437982a7'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:53:36.901165 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-b8d64787-fc1b-4925-92f5-e5be460bfbe2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8baae9fe605a48efa95c8be7437982a7'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:53:36.901912 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-b8d64787-fc1b-4925-92f5-e5be460bfbe2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8baae9fe605a48efa95c8be7437982a7'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:53:36.919790 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1267/2509] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Jul 27 10:53:36 2022] DELETE /identity/v3/users/8baae9fe605a48efa95c8be7437982a7 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:53:36.924245 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a19eec-4ab3-4ca1-bd6a-75c690bb8b90 None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:53:36.924747 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a19eec-4ab3-4ca1-bd6a-75c690bb8b90 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:53:36.924978 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a19eec-4ab3-4ca1-bd6a-75c690bb8b90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:53:36.945841 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-c1a19eec-4ab3-4ca1-bd6a-75c690bb8b90 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:53:36.947150 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-c1a19eec-4ab3-4ca1-bd6a-75c690bb8b90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:53:37.012864 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1243/2510] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:53:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:53:37.023237 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d64787-fc1b-4925-92f5-e5be460bfbe2 admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:53:37.026795 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-b8d64787-fc1b-4925-92f5-e5be460bfbe2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:53:37.102353 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:53:37.110297 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d64787-fc1b-4925-92f5-e5be460bfbe2 admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:53:37.115795 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4319fa7f-c962-475d-8ca4-81517cc91a49 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7922d0a1a7c4e808d0b2a80dca0da40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:53:37.116794 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4319fa7f-c962-475d-8ca4-81517cc91a49 service neutron] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:53:37.117431 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4319fa7f-c962-475d-8ca4-81517cc91a49 service neutron] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:53:37.117704 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4319fa7f-c962-475d-8ca4-81517cc91a49 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:53:37.128795 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4319fa7f-c962-475d-8ca4-81517cc91a49 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:53:37.130366 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4319fa7f-c962-475d-8ca4-81517cc91a49 service neutron] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:53:37.141986 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1268/2511] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:53:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 10:53:37.732080 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a19eec-4ab3-4ca1-bd6a-75c690bb8b90 None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:53:37.733506 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-c1a19eec-4ab3-4ca1-bd6a-75c690bb8b90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:53:37.745279 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a19eec-4ab3-4ca1-bd6a-75c690bb8b90 None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:53:37.750307 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9073c72d-0ca2-415b-88b5-ea78734a3a32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:53:37.750997 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9073c72d-0ca2-415b-88b5-ea78734a3a32 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:53:37.751469 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9073c72d-0ca2-415b-88b5-ea78734a3a32 admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:53:37.751569 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9073c72d-0ca2-415b-88b5-ea78734a3a32 admin admin] PATH_INFO: `/v3/projects/7695445cad964faeaa78f760628fdf5a` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:53:37.753183 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9073c72d-0ca2-415b-88b5-ea78734a3a32 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7695445cad964faeaa78f760628fdf5a)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:53:37.755169 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9073c72d-0ca2-415b-88b5-ea78734a3a32 admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:53:37.807125 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:53:37.811174 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-9073c72d-0ca2-415b-88b5-ea78734a3a32 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7695445cad964faeaa78f760628fdf5a', 'request_id': 'req-9073c72d-0ca2-415b-88b5-ea78734a3a32'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:53:37.817389 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-9073c72d-0ca2-415b-88b5-ea78734a3a32 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7695445cad964faeaa78f760628fdf5a', 'request_id': 'req-9073c72d-0ca2-415b-88b5-ea78734a3a32'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:53:37.825346 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-9073c72d-0ca2-415b-88b5-ea78734a3a32 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7695445cad964faeaa78f760628fdf5a', 'request_id': 'req-9073c72d-0ca2-415b-88b5-ea78734a3a32'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:53:37.835645 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-9073c72d-0ca2-415b-88b5-ea78734a3a32 admin admin] The token cache is being invalidate because project 7695445cad964faeaa78f760628fdf5a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86828) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 10:53:37.835958 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-9073c72d-0ca2-415b-88b5-ea78734a3a32 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:53:37.836634 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-9073c72d-0ca2-415b-88b5-ea78734a3a32 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:53:37.838305 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1244/2512] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Jul 27 10:53:37 2022] DELETE /identity/v3/projects/7695445cad964faeaa78f760628fdf5a => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:53:37.843175 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4319fa7f-c962-475d-8ca4-81517cc91a49 service neutron] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:53:37.853049 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4319fa7f-c962-475d-8ca4-81517cc91a49 service neutron] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:53:37.909055 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd402b9a-6036-4979-a4f9-20740d2034ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:53:37.909692 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd402b9a-6036-4979-a4f9-20740d2034ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:53:37.909981 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd402b9a-6036-4979-a4f9-20740d2034ce admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:53:37.910262 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd402b9a-6036-4979-a4f9-20740d2034ce admin admin] PATH_INFO: `/v3/users/1b882ffee1ca4cbfbf5c4310d6cc4fa5` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:53:37.911517 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd402b9a-6036-4979-a4f9-20740d2034ce admin admin] RBAC: Authorizing `identity:delete_user(user_id=1b882ffee1ca4cbfbf5c4310d6cc4fa5)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:53:37.913434 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd402b9a-6036-4979-a4f9-20740d2034ce admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:53:37.958812 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-cd402b9a-6036-4979-a4f9-20740d2034ce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1b882ffee1ca4cbfbf5c4310d6cc4fa5'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:53:37.966145 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-cd402b9a-6036-4979-a4f9-20740d2034ce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1b882ffee1ca4cbfbf5c4310d6cc4fa5'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:53:37.973586 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-cd402b9a-6036-4979-a4f9-20740d2034ce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1b882ffee1ca4cbfbf5c4310d6cc4fa5'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:53:37.974362 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-cd402b9a-6036-4979-a4f9-20740d2034ce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1b882ffee1ca4cbfbf5c4310d6cc4fa5'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:53:37.974988 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-cd402b9a-6036-4979-a4f9-20740d2034ce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1b882ffee1ca4cbfbf5c4310d6cc4fa5'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:53:37.991748 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1269/2513] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Jul 27 10:53:37 2022] DELETE /identity/v3/users/1b882ffee1ca4cbfbf5c4310d6cc4fa5 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:53:38.269721 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9073c72d-0ca2-415b-88b5-ea78734a3a32 admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:53:38.271272 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-9073c72d-0ca2-415b-88b5-ea78734a3a32 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:53:38.284996 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9073c72d-0ca2-415b-88b5-ea78734a3a32 admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:53:38.341624 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1ebe65-4735-48c0-a613-ffab438d3016 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:53:38.342268 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1ebe65-4735-48c0-a613-ffab438d3016 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:53:38.342553 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1ebe65-4735-48c0-a613-ffab438d3016 admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:53:38.342874 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1ebe65-4735-48c0-a613-ffab438d3016 admin admin] PATH_INFO: `/v3/projects/0c7b05f0a8f64d31bd84b6277dd79e56` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:53:38.344033 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1ebe65-4735-48c0-a613-ffab438d3016 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0c7b05f0a8f64d31bd84b6277dd79e56)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:53:38.346449 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1ebe65-4735-48c0-a613-ffab438d3016 admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:53:38.394943 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-3e1ebe65-4735-48c0-a613-ffab438d3016 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0c7b05f0a8f64d31bd84b6277dd79e56', 'request_id': 'req-3e1ebe65-4735-48c0-a613-ffab438d3016'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:53:38.401136 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-3e1ebe65-4735-48c0-a613-ffab438d3016 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0c7b05f0a8f64d31bd84b6277dd79e56', 'request_id': 'req-3e1ebe65-4735-48c0-a613-ffab438d3016'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:53:38.411904 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-3e1ebe65-4735-48c0-a613-ffab438d3016 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0c7b05f0a8f64d31bd84b6277dd79e56', 'request_id': 'req-3e1ebe65-4735-48c0-a613-ffab438d3016'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:53:38.423602 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-3e1ebe65-4735-48c0-a613-ffab438d3016 admin admin] The token cache is being invalidate because project 0c7b05f0a8f64d31bd84b6277dd79e56 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86828) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 10:53:38.423890 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-3e1ebe65-4735-48c0-a613-ffab438d3016 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:53:38.424520 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-3e1ebe65-4735-48c0-a613-ffab438d3016 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:53:38.426024 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1245/2514] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Jul 27 10:53:38 2022] DELETE /identity/v3/projects/0c7b05f0a8f64d31bd84b6277dd79e56 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:53:38.431097 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566caba5-2366-4869-b392-5a14b8b909fd None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:53:38.431394 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566caba5-2366-4869-b392-5a14b8b909fd None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:53:38.431676 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566caba5-2366-4869-b392-5a14b8b909fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:53:38.452545 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-566caba5-2366-4869-b392-5a14b8b909fd None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:53:38.453952 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-566caba5-2366-4869-b392-5a14b8b909fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:53:38.525465 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1270/2515] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:53:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:53:38.533386 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d4d988-32cc-472c-afd5-0611adb1c9ab None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:53:38.533683 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d4d988-32cc-472c-afd5-0611adb1c9ab None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:53:38.533966 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d4d988-32cc-472c-afd5-0611adb1c9ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:53:38.554231 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-29d4d988-32cc-472c-afd5-0611adb1c9ab None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:53:38.555477 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-29d4d988-32cc-472c-afd5-0611adb1c9ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:53:38.564152 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1246/2516] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:53:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 10:53:38.568492 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566caba5-2366-4869-b392-5a14b8b909fd None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:53:38.578218 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566caba5-2366-4869-b392-5a14b8b909fd None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:53:38.583034 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3aa7d3d-2fc9-4e61-a14f-837f7c76f41c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:53:38.583905 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3aa7d3d-2fc9-4e61-a14f-837f7c76f41c admin admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:53:38.584212 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3aa7d3d-2fc9-4e61-a14f-837f7c76f41c admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:53:38.584555 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3aa7d3d-2fc9-4e61-a14f-837f7c76f41c admin admin] PATH_INFO: `/v3/domains` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:53:38.585512 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3aa7d3d-2fc9-4e61-a14f-837f7c76f41c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86827) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:53:38.585881 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3aa7d3d-2fc9-4e61-a14f-837f7c76f41c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:53:38.587623 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3aa7d3d-2fc9-4e61-a14f-837f7c76f41c admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:53:38.590557 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:53:38.603800 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1271/2517] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Jul 27 10:53:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:53:38.611669 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d4d988-32cc-472c-afd5-0611adb1c9ab None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:53:38.623427 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d4d988-32cc-472c-afd5-0611adb1c9ab None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:53:38.629250 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86844f2b-f841-4684-9941-9f557e436dfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:53:38.629905 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86844f2b-f841-4684-9941-9f557e436dfd admin admin] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:53:38.630298 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86844f2b-f841-4684-9941-9f557e436dfd admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:53:38.630579 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86844f2b-f841-4684-9941-9f557e436dfd admin admin] PATH_INFO: `/v3/projects` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:53:38.631394 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86844f2b-f841-4684-9941-9f557e436dfd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:53:38.633467 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86844f2b-f841-4684-9941-9f557e436dfd admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:53:38.665911 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1247/2518] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Jul 27 10:53:38 2022] POST /identity/v3/projects => generated 381 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 10:53:38.668430 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3aa7d3d-2fc9-4e61-a14f-837f7c76f41c admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:53:38.679334 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3aa7d3d-2fc9-4e61-a14f-837f7c76f41c admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:53:38.740819 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3d4973-0bd1-4bc1-8a23-1e4791af5899 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:53:38.741376 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3d4973-0bd1-4bc1-8a23-1e4791af5899 admin admin] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:53:38.741663 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3d4973-0bd1-4bc1-8a23-1e4791af5899 admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:53:38.741937 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3d4973-0bd1-4bc1-8a23-1e4791af5899 admin admin] PATH_INFO: `/v3/users` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:53:38.742627 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3d4973-0bd1-4bc1-8a23-1e4791af5899 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:53:38.744561 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3d4973-0bd1-4bc1-8a23-1e4791af5899 admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:53:38.765569 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1272/2519] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Jul 27 10:53:38 2022] POST /identity/v3/users => generated 291 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 10:53:38.770612 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86844f2b-f841-4684-9941-9f557e436dfd admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:53:38.781923 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86844f2b-f841-4684-9941-9f557e436dfd admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:53:38.786661 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaeaf357-0d40-4062-80a5-595c1cc179a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:53:38.787289 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaeaf357-0d40-4062-80a5-595c1cc179a8 admin admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:53:38.787573 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaeaf357-0d40-4062-80a5-595c1cc179a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:53:38.787849 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaeaf357-0d40-4062-80a5-595c1cc179a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:53:38.788550 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaeaf357-0d40-4062-80a5-595c1cc179a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:53:38.790152 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaeaf357-0d40-4062-80a5-595c1cc179a8 admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:53:38.800989 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1248/2520] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 10:53:38 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 10:53:38.804949 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3d4973-0bd1-4bc1-8a23-1e4791af5899 admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:53:38.818156 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3d4973-0bd1-4bc1-8a23-1e4791af5899 admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:53:38.825572 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635cee89-535f-4ead-b6b4-cde5cd2d6012 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:53:38.826358 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635cee89-535f-4ead-b6b4-cde5cd2d6012 admin admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:53:38.826733 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635cee89-535f-4ead-b6b4-cde5cd2d6012 admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:53:38.827049 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635cee89-535f-4ead-b6b4-cde5cd2d6012 admin admin] PATH_INFO: `/v3/roles` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:53:38.828215 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635cee89-535f-4ead-b6b4-cde5cd2d6012 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:53:38.830976 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635cee89-535f-4ead-b6b4-cde5cd2d6012 admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:53:38.847816 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1273/2521] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 10:53:38 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 10:53:38.852058 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaeaf357-0d40-4062-80a5-595c1cc179a8 admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:53:38.862724 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaeaf357-0d40-4062-80a5-595c1cc179a8 admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:53:38.867609 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218dc93a-0c4b-42b8-acbb-a9dff9433b7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:53:38.868374 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218dc93a-0c4b-42b8-acbb-a9dff9433b7f admin admin] REQUEST_METHOD: `PUT` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:53:38.868599 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218dc93a-0c4b-42b8-acbb-a9dff9433b7f admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:53:38.868928 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218dc93a-0c4b-42b8-acbb-a9dff9433b7f admin admin] PATH_INFO: `/v3/projects/41b09fa782ee4e649fab3dc4a4665555/users/e860c7a32cb54ec0bd47bbd60125e238/roles/4c9fc0b3baed441da2126be41cc6b04a` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:53:38.884731 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218dc93a-0c4b-42b8-acbb-a9dff9433b7f admin admin] RBAC: Authorizing `identity:create_grant(project_id=41b09fa782ee4e649fab3dc4a4665555, user_id=e860c7a32cb54ec0bd47bbd60125e238, role_id=4c9fc0b3baed441da2126be41cc6b04a)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:53:38.887522 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218dc93a-0c4b-42b8-acbb-a9dff9433b7f admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:53:38.895939 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=85,keystone:DELETE=11,keystone:INSERT=4 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:53:38.902990 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1249/2522] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 10:53:38 2022] PUT /identity/v3/projects/41b09fa782ee4e649fab3dc4a4665555/users/e860c7a32cb54ec0bd47bbd60125e238/roles/4c9fc0b3baed441da2126be41cc6b04a => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:53:38.913012 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b053ebbf-ef6a-4afa-b626-39b2548bb2bf None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:53:38.913306 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b053ebbf-ef6a-4afa-b626-39b2548bb2bf None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:53:38.913572 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b053ebbf-ef6a-4afa-b626-39b2548bb2bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:53:38.942559 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-b053ebbf-ef6a-4afa-b626-39b2548bb2bf None None] MFA Rules not processed for user `e860c7a32cb54ec0bd47bbd60125e238`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:53:38.943830 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-b053ebbf-ef6a-4afa-b626-39b2548bb2bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:53:39.014129 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1274/2523] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:53:38 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:53:39.018645 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218dc93a-0c4b-42b8-acbb-a9dff9433b7f admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:53:39.084498 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218dc93a-0c4b-42b8-acbb-a9dff9433b7f admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:53:39.089298 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e1554b-583b-4f8a-b231-f7061bd225eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:53:39.090539 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e1554b-583b-4f8a-b231-f7061bd225eb admin admin] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:53:39.090856 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e1554b-583b-4f8a-b231-f7061bd225eb admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:53:39.091138 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e1554b-583b-4f8a-b231-f7061bd225eb admin admin] PATH_INFO: `/v3/projects` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:53:39.091808 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e1554b-583b-4f8a-b231-f7061bd225eb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:53:39.093746 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e1554b-583b-4f8a-b231-f7061bd225eb admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:53:39.120825 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1250/2524] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Jul 27 10:53:39 2022] POST /identity/v3/projects => generated 381 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 10:53:39.125242 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b053ebbf-ef6a-4afa-b626-39b2548bb2bf None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:53:39.135810 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b053ebbf-ef6a-4afa-b626-39b2548bb2bf None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:53:39.192979 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbbae66-4d3f-46c1-b196-d969f7aa47f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:53:39.193598 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbbae66-4d3f-46c1-b196-d969f7aa47f3 admin admin] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:53:39.193886 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbbae66-4d3f-46c1-b196-d969f7aa47f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:53:39.194175 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbbae66-4d3f-46c1-b196-d969f7aa47f3 admin admin] PATH_INFO: `/v3/users` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:53:39.194877 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbbae66-4d3f-46c1-b196-d969f7aa47f3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:53:39.196788 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbbae66-4d3f-46c1-b196-d969f7aa47f3 admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:53:39.206417 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:53:39.218023 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1275/2525] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Jul 27 10:53:39 2022] POST /identity/v3/users => generated 297 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 10:53:39.222081 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e1554b-583b-4f8a-b231-f7061bd225eb admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:53:39.232888 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e1554b-583b-4f8a-b231-f7061bd225eb admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:53:39.237622 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604a9e2b-f110-4973-a514-25c0862f6fa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:53:39.238237 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604a9e2b-f110-4973-a514-25c0862f6fa4 admin admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:53:39.238521 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604a9e2b-f110-4973-a514-25c0862f6fa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:53:39.238832 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604a9e2b-f110-4973-a514-25c0862f6fa4 admin admin] PATH_INFO: `/v3/roles` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:53:39.239529 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604a9e2b-f110-4973-a514-25c0862f6fa4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:53:39.241229 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604a9e2b-f110-4973-a514-25c0862f6fa4 admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:53:39.252667 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1251/2526] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 10:53:39 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 10:53:39.257810 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbbae66-4d3f-46c1-b196-d969f7aa47f3 admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:53:39.269798 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbbae66-4d3f-46c1-b196-d969f7aa47f3 admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:53:39.275216 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dddaa6c-d2ca-4939-81bf-05e65a51033b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:53:39.275878 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dddaa6c-d2ca-4939-81bf-05e65a51033b admin admin] REQUEST_METHOD: `PUT` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:53:39.276189 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dddaa6c-d2ca-4939-81bf-05e65a51033b admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:53:39.276463 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dddaa6c-d2ca-4939-81bf-05e65a51033b admin admin] PATH_INFO: `/v3/projects/c350b7fc840048dcad5998d103bf52c5/users/0a702302b4974dd5beef1555e3a8876d/roles/21156560eb6b436bb567a40b4976f3f4` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:53:39.291475 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dddaa6c-d2ca-4939-81bf-05e65a51033b admin admin] RBAC: Authorizing `identity:create_grant(project_id=c350b7fc840048dcad5998d103bf52c5, user_id=0a702302b4974dd5beef1555e3a8876d, role_id=21156560eb6b436bb567a40b4976f3f4)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:53:39.294670 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dddaa6c-d2ca-4939-81bf-05e65a51033b admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:53:39.306999 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1276/2527] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 10:53:39 2022] PUT /identity/v3/projects/c350b7fc840048dcad5998d103bf52c5/users/0a702302b4974dd5beef1555e3a8876d/roles/21156560eb6b436bb567a40b4976f3f4 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 10:53:39.311708 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604a9e2b-f110-4973-a514-25c0862f6fa4 admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:53:39.322622 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604a9e2b-f110-4973-a514-25c0862f6fa4 admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:53:39.383048 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a6988d-4d8d-41fc-8a72-cf6f592e75d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:53:39.383676 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a6988d-4d8d-41fc-8a72-cf6f592e75d2 admin admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:53:39.383963 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a6988d-4d8d-41fc-8a72-cf6f592e75d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:53:39.384240 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a6988d-4d8d-41fc-8a72-cf6f592e75d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:53:39.384938 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a6988d-4d8d-41fc-8a72-cf6f592e75d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:53:39.386536 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a6988d-4d8d-41fc-8a72-cf6f592e75d2 admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:53:39.396943 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1252/2528] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 10:53:39 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 10:53:39.401209 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dddaa6c-d2ca-4939-81bf-05e65a51033b admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:53:39.411590 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dddaa6c-d2ca-4939-81bf-05e65a51033b admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:53:39.416429 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358584a5-7c58-48e8-859f-f2e6bea1c2c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:53:39.417114 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358584a5-7c58-48e8-859f-f2e6bea1c2c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:53:39.417395 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358584a5-7c58-48e8-859f-f2e6bea1c2c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:53:39.417682 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358584a5-7c58-48e8-859f-f2e6bea1c2c9 admin admin] PATH_INFO: `/v3/projects/c350b7fc840048dcad5998d103bf52c5/users/0a702302b4974dd5beef1555e3a8876d/roles/21156560eb6b436bb567a40b4976f3f4` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:53:39.420240 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358584a5-7c58-48e8-859f-f2e6bea1c2c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c350b7fc840048dcad5998d103bf52c5, user_id=0a702302b4974dd5beef1555e3a8876d, role_id=21156560eb6b436bb567a40b4976f3f4)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:53:39.422949 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358584a5-7c58-48e8-859f-f2e6bea1c2c9 admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:53:39.434841 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1277/2529] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 10:53:39 2022] PUT /identity/v3/projects/c350b7fc840048dcad5998d103bf52c5/users/0a702302b4974dd5beef1555e3a8876d/roles/21156560eb6b436bb567a40b4976f3f4 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:53:39.446179 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ea9efd-013c-4a73-a890-4eb616c0d7aa None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:53:39.446473 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ea9efd-013c-4a73-a890-4eb616c0d7aa None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:53:39.446790 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ea9efd-013c-4a73-a890-4eb616c0d7aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:53:39.475502 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-b6ea9efd-013c-4a73-a890-4eb616c0d7aa None None] MFA Rules not processed for user `0a702302b4974dd5beef1555e3a8876d`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:53:39.476827 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-b6ea9efd-013c-4a73-a890-4eb616c0d7aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:53:39.538018 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:53:39.551064 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1253/2530] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:53:39 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:53:39.562087 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358584a5-7c58-48e8-859f-f2e6bea1c2c9 admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:53:39.564212 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-358584a5-7c58-48e8-859f-f2e6bea1c2c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:53:39.623269 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358584a5-7c58-48e8-859f-f2e6bea1c2c9 admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:53:39.627260 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c2e79c-680e-4cbd-87e1-95870595e058 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '864d35f268cf45148756109759bb2743', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:53:39.627884 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c2e79c-680e-4cbd-87e1-95870595e058 service cinder] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:53:39.628223 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c2e79c-680e-4cbd-87e1-95870595e058 service cinder] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:53:39.628509 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c2e79c-680e-4cbd-87e1-95870595e058 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:53:39.637666 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c2e79c-680e-4cbd-87e1-95870595e058 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:53:39.639135 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c2e79c-680e-4cbd-87e1-95870595e058 service cinder] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:53:39.648575 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1278/2531] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:53:39 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 10:53:40.075844 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ea9efd-013c-4a73-a890-4eb616c0d7aa None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:53:40.143803 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ea9efd-013c-4a73-a890-4eb616c0d7aa None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:53:40.149028 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a2200d-01ab-4cdf-942e-5e13131eb859 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:53:40.149679 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a2200d-01ab-4cdf-942e-5e13131eb859 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:53:40.149956 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a2200d-01ab-4cdf-942e-5e13131eb859 admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:53:40.150229 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a2200d-01ab-4cdf-942e-5e13131eb859 admin admin] PATH_INFO: `/v3/users/e860c7a32cb54ec0bd47bbd60125e238` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:53:40.151451 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a2200d-01ab-4cdf-942e-5e13131eb859 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e860c7a32cb54ec0bd47bbd60125e238)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:53:40.153396 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a2200d-01ab-4cdf-942e-5e13131eb859 admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:53:40.204192 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-b6a2200d-01ab-4cdf-942e-5e13131eb859 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e860c7a32cb54ec0bd47bbd60125e238'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:53:40.211029 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-b6a2200d-01ab-4cdf-942e-5e13131eb859 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e860c7a32cb54ec0bd47bbd60125e238'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:53:40.218535 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-b6a2200d-01ab-4cdf-942e-5e13131eb859 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e860c7a32cb54ec0bd47bbd60125e238'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:53:40.219180 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-b6a2200d-01ab-4cdf-942e-5e13131eb859 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e860c7a32cb54ec0bd47bbd60125e238'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:53:40.219781 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-b6a2200d-01ab-4cdf-942e-5e13131eb859 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e860c7a32cb54ec0bd47bbd60125e238'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:53:40.237665 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1254/2532] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Jul 27 10:53:40 2022] DELETE /identity/v3/users/e860c7a32cb54ec0bd47bbd60125e238 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:53:40.242129 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9e646c-b483-4d56-bed5-2647f2e7c3a5 None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:53:40.242627 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9e646c-b483-4d56-bed5-2647f2e7c3a5 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:53:40.242748 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9e646c-b483-4d56-bed5-2647f2e7c3a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:53:40.261578 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-2d9e646c-b483-4d56-bed5-2647f2e7c3a5 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:53:40.262804 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-2d9e646c-b483-4d56-bed5-2647f2e7c3a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:53:40.320253 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1279/2533] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:53:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:53:40.329657 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a2200d-01ab-4cdf-942e-5e13131eb859 admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:53:40.331091 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-b6a2200d-01ab-4cdf-942e-5e13131eb859 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:53:40.387602 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a2200d-01ab-4cdf-942e-5e13131eb859 admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:53:40.391359 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b65548-3af3-4fe0-951b-972523190be4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7922d0a1a7c4e808d0b2a80dca0da40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:53:40.391992 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b65548-3af3-4fe0-951b-972523190be4 service neutron] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:53:40.392265 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b65548-3af3-4fe0-951b-972523190be4 service neutron] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:53:40.392545 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b65548-3af3-4fe0-951b-972523190be4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:53:40.401835 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b65548-3af3-4fe0-951b-972523190be4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:53:40.403809 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b65548-3af3-4fe0-951b-972523190be4 service neutron] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:53:40.413484 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1255/2534] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:53:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 10:53:40.696035 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9e646c-b483-4d56-bed5-2647f2e7c3a5 None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:53:40.697467 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-2d9e646c-b483-4d56-bed5-2647f2e7c3a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:53:40.711704 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9e646c-b483-4d56-bed5-2647f2e7c3a5 None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:53:40.717548 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d235c795-fead-47fd-9d4b-d69a5805e3c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:53:40.718371 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d235c795-fead-47fd-9d4b-d69a5805e3c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:53:40.718699 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d235c795-fead-47fd-9d4b-d69a5805e3c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:53:40.718985 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d235c795-fead-47fd-9d4b-d69a5805e3c0 admin admin] PATH_INFO: `/v3/projects/41b09fa782ee4e649fab3dc4a4665555` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:53:40.720287 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d235c795-fead-47fd-9d4b-d69a5805e3c0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=41b09fa782ee4e649fab3dc4a4665555)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:53:40.722407 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d235c795-fead-47fd-9d4b-d69a5805e3c0 admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:53:40.772521 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-d235c795-fead-47fd-9d4b-d69a5805e3c0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '41b09fa782ee4e649fab3dc4a4665555', 'request_id': 'req-d235c795-fead-47fd-9d4b-d69a5805e3c0'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:53:40.780391 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-d235c795-fead-47fd-9d4b-d69a5805e3c0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '41b09fa782ee4e649fab3dc4a4665555', 'request_id': 'req-d235c795-fead-47fd-9d4b-d69a5805e3c0'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:53:40.782278 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:INSERT=3,keystone:SELECT=93,keystone:DELETE=4 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:53:40.795341 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-d235c795-fead-47fd-9d4b-d69a5805e3c0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '41b09fa782ee4e649fab3dc4a4665555', 'request_id': 'req-d235c795-fead-47fd-9d4b-d69a5805e3c0'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:53:40.811960 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-d235c795-fead-47fd-9d4b-d69a5805e3c0 admin admin] The token cache is being invalidate because project 41b09fa782ee4e649fab3dc4a4665555 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86827) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 10:53:40.812339 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-d235c795-fead-47fd-9d4b-d69a5805e3c0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:53:40.812986 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-d235c795-fead-47fd-9d4b-d69a5805e3c0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:53:40.814521 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1280/2535] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Jul 27 10:53:40 2022] DELETE /identity/v3/projects/41b09fa782ee4e649fab3dc4a4665555 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 10:53:40.818797 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b65548-3af3-4fe0-951b-972523190be4 service neutron] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:53:40.830308 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b65548-3af3-4fe0-951b-972523190be4 service neutron] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:53:40.842983 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:53:40.890460 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27cb84b-a003-48d0-8b55-2a86e04492ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:53:40.891070 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27cb84b-a003-48d0-8b55-2a86e04492ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:53:40.891358 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27cb84b-a003-48d0-8b55-2a86e04492ff admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:53:40.891637 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27cb84b-a003-48d0-8b55-2a86e04492ff admin admin] PATH_INFO: `/v3/users/0a702302b4974dd5beef1555e3a8876d` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:53:40.892798 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27cb84b-a003-48d0-8b55-2a86e04492ff admin admin] RBAC: Authorizing `identity:delete_user(user_id=0a702302b4974dd5beef1555e3a8876d)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:53:40.894827 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27cb84b-a003-48d0-8b55-2a86e04492ff admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:53:40.939173 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-b27cb84b-a003-48d0-8b55-2a86e04492ff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0a702302b4974dd5beef1555e3a8876d'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:53:40.946337 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-b27cb84b-a003-48d0-8b55-2a86e04492ff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0a702302b4974dd5beef1555e3a8876d'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:53:40.954628 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-b27cb84b-a003-48d0-8b55-2a86e04492ff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0a702302b4974dd5beef1555e3a8876d'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:53:40.955335 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-b27cb84b-a003-48d0-8b55-2a86e04492ff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0a702302b4974dd5beef1555e3a8876d'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:53:40.956146 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-b27cb84b-a003-48d0-8b55-2a86e04492ff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0a702302b4974dd5beef1555e3a8876d'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:53:40.975137 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1256/2536] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Jul 27 10:53:40 2022] DELETE /identity/v3/users/0a702302b4974dd5beef1555e3a8876d => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:53:41.332502 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d235c795-fead-47fd-9d4b-d69a5805e3c0 admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:53:41.334570 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-d235c795-fead-47fd-9d4b-d69a5805e3c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:53:41.362461 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d235c795-fead-47fd-9d4b-d69a5805e3c0 admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:53:41.445990 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9c9028-8644-40e1-b9b2-b68e3ddf3377 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:53:41.446680 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9c9028-8644-40e1-b9b2-b68e3ddf3377 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:53:41.446971 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9c9028-8644-40e1-b9b2-b68e3ddf3377 admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:53:41.447252 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9c9028-8644-40e1-b9b2-b68e3ddf3377 admin admin] PATH_INFO: `/v3/projects/c350b7fc840048dcad5998d103bf52c5` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:53:41.448421 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9c9028-8644-40e1-b9b2-b68e3ddf3377 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c350b7fc840048dcad5998d103bf52c5)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:53:41.450971 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9c9028-8644-40e1-b9b2-b68e3ddf3377 admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:53:41.504486 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-bd9c9028-8644-40e1-b9b2-b68e3ddf3377 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c350b7fc840048dcad5998d103bf52c5', 'request_id': 'req-bd9c9028-8644-40e1-b9b2-b68e3ddf3377'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:53:41.511170 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-bd9c9028-8644-40e1-b9b2-b68e3ddf3377 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c350b7fc840048dcad5998d103bf52c5', 'request_id': 'req-bd9c9028-8644-40e1-b9b2-b68e3ddf3377'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:53:41.520741 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-bd9c9028-8644-40e1-b9b2-b68e3ddf3377 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c350b7fc840048dcad5998d103bf52c5', 'request_id': 'req-bd9c9028-8644-40e1-b9b2-b68e3ddf3377'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:53:41.533537 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-bd9c9028-8644-40e1-b9b2-b68e3ddf3377 admin admin] The token cache is being invalidate because project c350b7fc840048dcad5998d103bf52c5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86827) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 10:53:41.533895 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-bd9c9028-8644-40e1-b9b2-b68e3ddf3377 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:53:41.534501 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-bd9c9028-8644-40e1-b9b2-b68e3ddf3377 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:53:41.536337 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1281/2537] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Jul 27 10:53:41 2022] DELETE /identity/v3/projects/c350b7fc840048dcad5998d103bf52c5 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:53:41.541755 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43eab7d4-e066-4663-aa5d-8bfbece9dc44 None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:53:41.542296 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43eab7d4-e066-4663-aa5d-8bfbece9dc44 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:53:41.542529 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43eab7d4-e066-4663-aa5d-8bfbece9dc44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:53:41.565018 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-43eab7d4-e066-4663-aa5d-8bfbece9dc44 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:53:41.566231 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-43eab7d4-e066-4663-aa5d-8bfbece9dc44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:53:41.627303 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1257/2538] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:53:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:53:41.635253 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92f13f6-cb29-4e7d-945c-81c81fb942c5 None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:53:41.635690 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92f13f6-cb29-4e7d-945c-81c81fb942c5 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:53:41.635976 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92f13f6-cb29-4e7d-945c-81c81fb942c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:53:41.657119 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-b92f13f6-cb29-4e7d-945c-81c81fb942c5 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:53:41.658247 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-b92f13f6-cb29-4e7d-945c-81c81fb942c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:53:41.666222 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1282/2539] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:53:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:53:41.670682 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43eab7d4-e066-4663-aa5d-8bfbece9dc44 None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:53:41.680510 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43eab7d4-e066-4663-aa5d-8bfbece9dc44 None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:53:41.685265 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205ca266-96ae-4bb2-80fb-22f8cecd19e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:53:41.685887 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205ca266-96ae-4bb2-80fb-22f8cecd19e9 admin admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:53:41.686168 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205ca266-96ae-4bb2-80fb-22f8cecd19e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:53:41.686443 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205ca266-96ae-4bb2-80fb-22f8cecd19e9 admin admin] PATH_INFO: `/v3/domains` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:53:41.687223 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205ca266-96ae-4bb2-80fb-22f8cecd19e9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86828) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:53:41.687531 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205ca266-96ae-4bb2-80fb-22f8cecd19e9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:53:41.689141 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205ca266-96ae-4bb2-80fb-22f8cecd19e9 admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:53:41.702208 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1258/2540] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Jul 27 10:53:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 10:53:41.706877 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92f13f6-cb29-4e7d-945c-81c81fb942c5 None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:53:41.716923 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92f13f6-cb29-4e7d-945c-81c81fb942c5 None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:53:41.721624 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7c4df1-7cc7-42d3-b9ce-a26fa21fb69f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:53:41.722228 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7c4df1-7cc7-42d3-b9ce-a26fa21fb69f admin admin] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:53:41.722505 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7c4df1-7cc7-42d3-b9ce-a26fa21fb69f admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:53:41.722814 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7c4df1-7cc7-42d3-b9ce-a26fa21fb69f admin admin] PATH_INFO: `/v3/projects` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:53:41.723481 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7c4df1-7cc7-42d3-b9ce-a26fa21fb69f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:53:41.725368 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7c4df1-7cc7-42d3-b9ce-a26fa21fb69f admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:53:41.752972 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1283/2541] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Jul 27 10:53:41 2022] POST /identity/v3/projects => generated 381 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 10:53:41.757083 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205ca266-96ae-4bb2-80fb-22f8cecd19e9 admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:53:41.768238 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205ca266-96ae-4bb2-80fb-22f8cecd19e9 admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:53:41.786750 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:53:41.827292 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf8923d-a68b-45e0-aa87-069c221defc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:53:41.827962 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf8923d-a68b-45e0-aa87-069c221defc2 admin admin] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:53:41.828279 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf8923d-a68b-45e0-aa87-069c221defc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:53:41.828562 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf8923d-a68b-45e0-aa87-069c221defc2 admin admin] PATH_INFO: `/v3/users` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:53:41.829250 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf8923d-a68b-45e0-aa87-069c221defc2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:53:41.831192 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf8923d-a68b-45e0-aa87-069c221defc2 admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:53:41.851711 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1259/2542] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Jul 27 10:53:41 2022] POST /identity/v3/users => generated 291 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 10:53:41.855841 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7c4df1-7cc7-42d3-b9ce-a26fa21fb69f admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:53:41.866436 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7c4df1-7cc7-42d3-b9ce-a26fa21fb69f admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:53:41.871607 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0653b805-103b-46a2-8237-6bbc8ad3c49b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:53:41.872218 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0653b805-103b-46a2-8237-6bbc8ad3c49b admin admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:53:41.872504 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0653b805-103b-46a2-8237-6bbc8ad3c49b admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:53:41.872781 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0653b805-103b-46a2-8237-6bbc8ad3c49b admin admin] PATH_INFO: `/v3/roles` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:53:41.873494 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0653b805-103b-46a2-8237-6bbc8ad3c49b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:53:41.875410 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0653b805-103b-46a2-8237-6bbc8ad3c49b admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:53:41.886461 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1284/2543] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 10:53:41 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 10:53:41.890969 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf8923d-a68b-45e0-aa87-069c221defc2 admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:53:41.900780 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf8923d-a68b-45e0-aa87-069c221defc2 admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:53:41.905544 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a2061b-8c02-431f-90e2-fd49e4feb19e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:53:41.906152 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a2061b-8c02-431f-90e2-fd49e4feb19e admin admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:53:41.906440 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a2061b-8c02-431f-90e2-fd49e4feb19e admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:53:41.906764 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a2061b-8c02-431f-90e2-fd49e4feb19e admin admin] PATH_INFO: `/v3/roles` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:53:41.907511 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a2061b-8c02-431f-90e2-fd49e4feb19e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:53:41.909062 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a2061b-8c02-431f-90e2-fd49e4feb19e admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:53:41.919587 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1260/2544] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 10:53:41 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 10:53:41.923523 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0653b805-103b-46a2-8237-6bbc8ad3c49b admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:53:41.933804 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0653b805-103b-46a2-8237-6bbc8ad3c49b admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:53:41.938513 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8f29da-a2e9-4d06-820d-cfea4ff5d19c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:53:41.939202 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8f29da-a2e9-4d06-820d-cfea4ff5d19c admin admin] REQUEST_METHOD: `PUT` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:53:41.939486 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8f29da-a2e9-4d06-820d-cfea4ff5d19c admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:53:41.939764 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8f29da-a2e9-4d06-820d-cfea4ff5d19c admin admin] PATH_INFO: `/v3/projects/6b50dc16d14d4b869f1a6555db6c8373/users/0621626f2863400c903de6a88da84f1e/roles/4c9fc0b3baed441da2126be41cc6b04a` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:53:41.955130 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8f29da-a2e9-4d06-820d-cfea4ff5d19c admin admin] RBAC: Authorizing `identity:create_grant(project_id=6b50dc16d14d4b869f1a6555db6c8373, user_id=0621626f2863400c903de6a88da84f1e, role_id=4c9fc0b3baed441da2126be41cc6b04a)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:53:41.957770 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8f29da-a2e9-4d06-820d-cfea4ff5d19c admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:53:41.971345 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1285/2545] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 10:53:41 2022] PUT /identity/v3/projects/6b50dc16d14d4b869f1a6555db6c8373/users/0621626f2863400c903de6a88da84f1e/roles/4c9fc0b3baed441da2126be41cc6b04a => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:53:41.976369 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf94df1-799a-4bed-bf3d-8e281da5c3d9 None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:53:41.976660 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf94df1-799a-4bed-bf3d-8e281da5c3d9 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:53:41.976936 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf94df1-799a-4bed-bf3d-8e281da5c3d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:53:41.997269 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-8bf94df1-799a-4bed-bf3d-8e281da5c3d9 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:53:41.998581 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-8bf94df1-799a-4bed-bf3d-8e281da5c3d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:53:42.056095 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1261/2546] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:53:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:53:42.066096 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8f29da-a2e9-4d06-820d-cfea4ff5d19c admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:53:42.067838 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-cc8f29da-a2e9-4d06-820d-cfea4ff5d19c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:53:42.079428 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:INSERT=4,keystone:DELETE=10,keystone:SELECT=86 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:53:42.127216 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8f29da-a2e9-4d06-820d-cfea4ff5d19c admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:53:42.130911 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf4ea15-add3-49d0-aa38-fddfef0556e4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7922d0a1a7c4e808d0b2a80dca0da40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:53:42.131528 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf4ea15-add3-49d0-aa38-fddfef0556e4 service neutron] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:53:42.131812 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf4ea15-add3-49d0-aa38-fddfef0556e4 service neutron] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:53:42.132090 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf4ea15-add3-49d0-aa38-fddfef0556e4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:53:42.140999 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf4ea15-add3-49d0-aa38-fddfef0556e4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:53:42.142406 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf4ea15-add3-49d0-aa38-fddfef0556e4 service neutron] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:53:42.151838 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1286/2547] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:53:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 10:53:42.930358 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6f068b-9cf7-4eac-8705-96fe042b7b28 None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:53:42.930912 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6f068b-9cf7-4eac-8705-96fe042b7b28 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:53:42.931074 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6f068b-9cf7-4eac-8705-96fe042b7b28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:53:42.962402 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-5f6f068b-9cf7-4eac-8705-96fe042b7b28 None None] MFA Rules not processed for user `0621626f2863400c903de6a88da84f1e`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:53:42.963692 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-5f6f068b-9cf7-4eac-8705-96fe042b7b28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:53:43.032151 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1262/2548] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:53:42 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:53:43.043076 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf4ea15-add3-49d0-aa38-fddfef0556e4 service neutron] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:53:43.044534 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-3bf4ea15-add3-49d0-aa38-fddfef0556e4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:53:43.102552 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf4ea15-add3-49d0-aa38-fddfef0556e4 service neutron] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:53:43.106304 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b448e8-c1a4-4d1d-a648-c3cb29485c64 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '864d35f268cf45148756109759bb2743', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:53:43.106954 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b448e8-c1a4-4d1d-a648-c3cb29485c64 service cinder] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:53:43.107247 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b448e8-c1a4-4d1d-a648-c3cb29485c64 service cinder] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:53:43.107527 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b448e8-c1a4-4d1d-a648-c3cb29485c64 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:53:43.115894 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b448e8-c1a4-4d1d-a648-c3cb29485c64 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:53:43.117318 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b448e8-c1a4-4d1d-a648-c3cb29485c64 service cinder] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:53:43.126335 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1287/2549] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:53:43 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 10:53:44.117560 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6f068b-9cf7-4eac-8705-96fe042b7b28 None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:53:44.119652 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-5f6f068b-9cf7-4eac-8705-96fe042b7b28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:53:44.167488 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:53:44.223173 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6f068b-9cf7-4eac-8705-96fe042b7b28 None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:53:44.231823 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e1e1e4-f0ed-4425-b628-669a0c3fdd7a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e059d8b6c28c4fcba925d05d7c98e4ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:53:44.232714 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e1e1e4-f0ed-4425-b628-669a0c3fdd7a service nova] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:53:44.232992 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e1e1e4-f0ed-4425-b628-669a0c3fdd7a service nova] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:53:44.233267 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e1e1e4-f0ed-4425-b628-669a0c3fdd7a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:53:44.235220 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-e4e1e1e4-f0ed-4425-b628-669a0c3fdd7a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:53:44.250135 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e1e1e4-f0ed-4425-b628-669a0c3fdd7a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:53:44.252147 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e1e1e4-f0ed-4425-b628-669a0c3fdd7a service nova] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:53:44.266133 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1263/2550] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:53:44 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 10:53:44.488061 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11aad185-871c-490f-9d88-ab3d58093161 None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:53:44.488951 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11aad185-871c-490f-9d88-ab3d58093161 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:53:44.489235 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11aad185-871c-490f-9d88-ab3d58093161 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:53:44.508639 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-11aad185-871c-490f-9d88-ab3d58093161 None None] MFA Rules not processed for user `0621626f2863400c903de6a88da84f1e`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:53:44.509869 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-11aad185-871c-490f-9d88-ab3d58093161 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:53:44.518119 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1288/2551] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:53:44 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:53:44.765349 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e1e1e4-f0ed-4425-b628-669a0c3fdd7a service nova] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:53:44.766939 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-e4e1e1e4-f0ed-4425-b628-669a0c3fdd7a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:53:44.779132 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e1e1e4-f0ed-4425-b628-669a0c3fdd7a service nova] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:53:44.783193 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1ad8b6-d4ec-4335-ae2d-055c6aada024 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7922d0a1a7c4e808d0b2a80dca0da40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:53:44.783890 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1ad8b6-d4ec-4335-ae2d-055c6aada024 service neutron] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:53:44.784252 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1ad8b6-d4ec-4335-ae2d-055c6aada024 service neutron] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:53:44.784591 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1ad8b6-d4ec-4335-ae2d-055c6aada024 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:53:44.786396 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-ab1ad8b6-d4ec-4335-ae2d-055c6aada024 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:53:44.796951 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1ad8b6-d4ec-4335-ae2d-055c6aada024 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:53:44.798545 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1ad8b6-d4ec-4335-ae2d-055c6aada024 service neutron] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:53:44.807094 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1264/2552] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:53:44 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 10:53:45.224793 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11aad185-871c-490f-9d88-ab3d58093161 None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:53:45.226287 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-11aad185-871c-490f-9d88-ab3d58093161 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:53:45.298822 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11aad185-871c-490f-9d88-ab3d58093161 None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:53:45.305813 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e963b27d-09b1-41b0-a65f-cda2a1a64873 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16cc6179e9a04b6398fbacb40ad1c555', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:53:45.306440 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e963b27d-09b1-41b0-a65f-cda2a1a64873 service placement] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:53:45.306773 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e963b27d-09b1-41b0-a65f-cda2a1a64873 service placement] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:53:45.307053 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e963b27d-09b1-41b0-a65f-cda2a1a64873 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:53:45.309182 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-e963b27d-09b1-41b0-a65f-cda2a1a64873 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:53:45.321095 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e963b27d-09b1-41b0-a65f-cda2a1a64873 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:53:45.322780 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e963b27d-09b1-41b0-a65f-cda2a1a64873 service placement] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:53:45.335730 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1289/2553] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:53:45 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 10:53:47.164424 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1ad8b6-d4ec-4335-ae2d-055c6aada024 service neutron] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:53:47.173448 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1ad8b6-d4ec-4335-ae2d-055c6aada024 service neutron] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:53:47.177254 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab963432-435a-4449-a04a-4b03d7fc8b01 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7922d0a1a7c4e808d0b2a80dca0da40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:53:47.177866 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab963432-435a-4449-a04a-4b03d7fc8b01 service neutron] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:53:47.178150 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab963432-435a-4449-a04a-4b03d7fc8b01 service neutron] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:53:47.178437 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab963432-435a-4449-a04a-4b03d7fc8b01 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:53:47.180287 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-ab963432-435a-4449-a04a-4b03d7fc8b01 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:53:47.190246 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab963432-435a-4449-a04a-4b03d7fc8b01 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:53:47.191701 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab963432-435a-4449-a04a-4b03d7fc8b01 service neutron] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:53:47.200310 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1265/2554] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:53:47 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 10:53:49.425620 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e963b27d-09b1-41b0-a65f-cda2a1a64873 service placement] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:53:49.434965 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e963b27d-09b1-41b0-a65f-cda2a1a64873 service placement] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:53:49.438502 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f690a7-5774-4e3c-a845-b238f8e2ff0e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7922d0a1a7c4e808d0b2a80dca0da40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:53:49.439140 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f690a7-5774-4e3c-a845-b238f8e2ff0e service neutron] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:53:49.439427 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f690a7-5774-4e3c-a845-b238f8e2ff0e service neutron] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:53:49.439699 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f690a7-5774-4e3c-a845-b238f8e2ff0e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:53:49.441567 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-97f690a7-5774-4e3c-a845-b238f8e2ff0e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:53:49.453523 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f690a7-5774-4e3c-a845-b238f8e2ff0e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:53:49.454992 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f690a7-5774-4e3c-a845-b238f8e2ff0e service neutron] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:53:49.465022 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1290/2555] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:53:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 10:53:50.319746 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8fedf7-c125-41a3-aa41-3b51b01236a3 None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:53:50.320233 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8fedf7-c125-41a3-aa41-3b51b01236a3 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:53:50.320331 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8fedf7-c125-41a3-aa41-3b51b01236a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:53:50.341626 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-0d8fedf7-c125-41a3-aa41-3b51b01236a3 None None] MFA Rules not processed for user `3d9fed24b4ba44b0b5961a603c4362c4`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:53:50.342861 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-0d8fedf7-c125-41a3-aa41-3b51b01236a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:53:50.401965 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1266/2556] 10.222.0.36 () {58 vars in 1009 bytes} [Wed Jul 27 10:53:50 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:53:50.454403 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f690a7-5774-4e3c-a845-b238f8e2ff0e service neutron] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:53:50.455999 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-97f690a7-5774-4e3c-a845-b238f8e2ff0e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:53:50.491861 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=100 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:53:50.547277 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f690a7-5774-4e3c-a845-b238f8e2ff0e service neutron] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:53:50.553052 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271dd209-ff86-4237-9edc-f42eabae8246 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89ac506eefcd4e42a84b44fa15fb3975', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:53:50.553661 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271dd209-ff86-4237-9edc-f42eabae8246 service swift] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:53:50.553944 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271dd209-ff86-4237-9edc-f42eabae8246 service swift] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:53:50.554252 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271dd209-ff86-4237-9edc-f42eabae8246 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:53:50.563680 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271dd209-ff86-4237-9edc-f42eabae8246 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:53:50.565200 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271dd209-ff86-4237-9edc-f42eabae8246 service swift] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:53:50.573360 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1291/2557] 10.222.0.36 () {60 vars in 1384 bytes} [Wed Jul 27 10:53:50 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 27 10:53:53.157201 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8fedf7-c125-41a3-aa41-3b51b01236a3 None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:53:53.158826 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-0d8fedf7-c125-41a3-aa41-3b51b01236a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:53:53.190518 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8fedf7-c125-41a3-aa41-3b51b01236a3 None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:53:53.202518 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bfc65d-a680-450a-a905-dc631adff2e2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e059d8b6c28c4fcba925d05d7c98e4ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:53:53.206072 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bfc65d-a680-450a-a905-dc631adff2e2 service nova] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:53:53.207553 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bfc65d-a680-450a-a905-dc631adff2e2 service nova] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:53:53.208214 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bfc65d-a680-450a-a905-dc631adff2e2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:53:53.211742 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-d0bfc65d-a680-450a-a905-dc631adff2e2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:53:53.232823 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bfc65d-a680-450a-a905-dc631adff2e2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:53:53.235125 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bfc65d-a680-450a-a905-dc631adff2e2 service nova] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:53:53.266329 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1267/2558] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:53:53 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 10:53:53.345242 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271dd209-ff86-4237-9edc-f42eabae8246 service swift] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:53:53.347275 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-271dd209-ff86-4237-9edc-f42eabae8246 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:53:53.382308 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271dd209-ff86-4237-9edc-f42eabae8246 service swift] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:53:53.397506 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b02cb6f-aae9-46ea-814d-7e1109573aa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:53:53.397927 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b02cb6f-aae9-46ea-814d-7e1109573aa5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:53:53.398420 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b02cb6f-aae9-46ea-814d-7e1109573aa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:53:53.398899 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b02cb6f-aae9-46ea-814d-7e1109573aa5 admin admin] PATH_INFO: `/v3/users/9b094c00959246f5a113f600fb07215d` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:53:53.402269 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b02cb6f-aae9-46ea-814d-7e1109573aa5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9b094c00959246f5a113f600fb07215d)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:53:53.407294 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b02cb6f-aae9-46ea-814d-7e1109573aa5 admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:53:53.509253 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-4b02cb6f-aae9-46ea-814d-7e1109573aa5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9b094c00959246f5a113f600fb07215d'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:53:53.521197 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-4b02cb6f-aae9-46ea-814d-7e1109573aa5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9b094c00959246f5a113f600fb07215d'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:53:53.538422 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-4b02cb6f-aae9-46ea-814d-7e1109573aa5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9b094c00959246f5a113f600fb07215d'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:53:53.539662 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-4b02cb6f-aae9-46ea-814d-7e1109573aa5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9b094c00959246f5a113f600fb07215d'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:53:53.540669 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-4b02cb6f-aae9-46ea-814d-7e1109573aa5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9b094c00959246f5a113f600fb07215d'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:53:53.577393 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1292/2559] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Jul 27 10:53:53 2022] DELETE /identity/v3/users/9b094c00959246f5a113f600fb07215d => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:53:53.801888 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bfc65d-a680-450a-a905-dc631adff2e2 service nova] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:53:53.825333 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bfc65d-a680-450a-a905-dc631adff2e2 service nova] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:53:53.909512 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d083a037-baed-4c4a-9d26-7b1ad37b5c1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:53:53.910405 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d083a037-baed-4c4a-9d26-7b1ad37b5c1a admin admin] REQUEST_METHOD: `DELETE` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:53:53.910808 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d083a037-baed-4c4a-9d26-7b1ad37b5c1a admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:53:53.911093 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d083a037-baed-4c4a-9d26-7b1ad37b5c1a admin admin] PATH_INFO: `/v3/projects/c4f3fa51a14844b1babdbd9533b7c3ec` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:53:53.913658 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d083a037-baed-4c4a-9d26-7b1ad37b5c1a admin admin] RBAC: Authorizing `identity:delete_project(project_id=c4f3fa51a14844b1babdbd9533b7c3ec)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:53:53.919659 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d083a037-baed-4c4a-9d26-7b1ad37b5c1a admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:53:54.003526 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:53:54.040020 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-d083a037-baed-4c4a-9d26-7b1ad37b5c1a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c4f3fa51a14844b1babdbd9533b7c3ec', 'request_id': 'req-d083a037-baed-4c4a-9d26-7b1ad37b5c1a'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:53:54.054943 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-d083a037-baed-4c4a-9d26-7b1ad37b5c1a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c4f3fa51a14844b1babdbd9533b7c3ec', 'request_id': 'req-d083a037-baed-4c4a-9d26-7b1ad37b5c1a'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:53:54.067608 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-d083a037-baed-4c4a-9d26-7b1ad37b5c1a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c4f3fa51a14844b1babdbd9533b7c3ec', 'request_id': 'req-d083a037-baed-4c4a-9d26-7b1ad37b5c1a'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:53:54.085082 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-d083a037-baed-4c4a-9d26-7b1ad37b5c1a admin admin] The token cache is being invalidate because project c4f3fa51a14844b1babdbd9533b7c3ec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86828) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 10:53:54.085660 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-d083a037-baed-4c4a-9d26-7b1ad37b5c1a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:53:54.086476 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-d083a037-baed-4c4a-9d26-7b1ad37b5c1a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:53:54.088555 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1268/2560] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Jul 27 10:53:53 2022] DELETE /identity/v3/projects/c4f3fa51a14844b1babdbd9533b7c3ec => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:53:54.093889 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5aa4999-b43b-47e3-bf69-1d37f99fe1ba None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:53:54.094556 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5aa4999-b43b-47e3-bf69-1d37f99fe1ba None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:53:54.095476 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5aa4999-b43b-47e3-bf69-1d37f99fe1ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:53:54.133946 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-a5aa4999-b43b-47e3-bf69-1d37f99fe1ba None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:53:54.135450 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-a5aa4999-b43b-47e3-bf69-1d37f99fe1ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:53:54.216759 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1293/2561] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:53:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:53:54.225234 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de991983-2fdd-482f-a40e-55055ee69db9 None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:53:54.226162 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de991983-2fdd-482f-a40e-55055ee69db9 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:53:54.226450 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de991983-2fdd-482f-a40e-55055ee69db9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:53:54.248370 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-de991983-2fdd-482f-a40e-55055ee69db9 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:53:54.250166 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-de991983-2fdd-482f-a40e-55055ee69db9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:53:54.258778 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1269/2562] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:53:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:53:54.264028 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5aa4999-b43b-47e3-bf69-1d37f99fe1ba None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:53:54.274517 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5aa4999-b43b-47e3-bf69-1d37f99fe1ba None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:53:54.280265 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d81def5-4bbc-44af-bc7f-4134ff8a3a1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:53:54.280926 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d81def5-4bbc-44af-bc7f-4134ff8a3a1c admin admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:53:54.281216 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d81def5-4bbc-44af-bc7f-4134ff8a3a1c admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:53:54.281498 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d81def5-4bbc-44af-bc7f-4134ff8a3a1c admin admin] PATH_INFO: `/v3/domains` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:53:54.282227 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d81def5-4bbc-44af-bc7f-4134ff8a3a1c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86827) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:53:54.282538 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d81def5-4bbc-44af-bc7f-4134ff8a3a1c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:53:54.284267 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d81def5-4bbc-44af-bc7f-4134ff8a3a1c admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:53:54.301091 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1294/2563] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Jul 27 10:53:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:53:54.305521 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de991983-2fdd-482f-a40e-55055ee69db9 None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:53:54.321224 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de991983-2fdd-482f-a40e-55055ee69db9 None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:53:54.326787 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a292208-a3c4-4dc3-bab4-e3971c0eb5f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:53:54.327434 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a292208-a3c4-4dc3-bab4-e3971c0eb5f1 admin admin] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:53:54.327720 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a292208-a3c4-4dc3-bab4-e3971c0eb5f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:53:54.327999 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a292208-a3c4-4dc3-bab4-e3971c0eb5f1 admin admin] PATH_INFO: `/v3/projects` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:53:54.328676 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a292208-a3c4-4dc3-bab4-e3971c0eb5f1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:53:54.330802 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a292208-a3c4-4dc3-bab4-e3971c0eb5f1 admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:53:54.412505 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1270/2564] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Jul 27 10:53:54 2022] POST /identity/v3/projects => generated 385 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 10:53:54.416875 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d81def5-4bbc-44af-bc7f-4134ff8a3a1c admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:53:54.427712 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d81def5-4bbc-44af-bc7f-4134ff8a3a1c admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:53:54.437672 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:53:54.491554 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3415a09b-12f9-46aa-be02-3c850817a31f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:53:54.492296 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3415a09b-12f9-46aa-be02-3c850817a31f admin admin] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:53:54.492622 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3415a09b-12f9-46aa-be02-3c850817a31f admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:53:54.492984 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3415a09b-12f9-46aa-be02-3c850817a31f admin admin] PATH_INFO: `/v3/users` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:53:54.493722 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3415a09b-12f9-46aa-be02-3c850817a31f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:53:54.496443 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3415a09b-12f9-46aa-be02-3c850817a31f admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:53:54.520874 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1295/2565] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Jul 27 10:53:54 2022] POST /identity/v3/users => generated 293 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 10:53:54.525025 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a292208-a3c4-4dc3-bab4-e3971c0eb5f1 admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:53:54.535600 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a292208-a3c4-4dc3-bab4-e3971c0eb5f1 admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:53:54.540570 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c67aad-2204-4164-bd05-8584a6fbf8b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:53:54.541208 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c67aad-2204-4164-bd05-8584a6fbf8b2 admin admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:53:54.541488 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c67aad-2204-4164-bd05-8584a6fbf8b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:53:54.541758 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c67aad-2204-4164-bd05-8584a6fbf8b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:53:54.542642 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c67aad-2204-4164-bd05-8584a6fbf8b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:53:54.544245 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c67aad-2204-4164-bd05-8584a6fbf8b2 admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:53:54.558647 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1271/2566] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 10:53:54 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 10:53:54.561261 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3415a09b-12f9-46aa-be02-3c850817a31f admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:53:54.571877 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3415a09b-12f9-46aa-be02-3c850817a31f admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:53:54.577226 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377b338d-8417-466a-a5e2-8a0e62d57c45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:53:54.577852 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377b338d-8417-466a-a5e2-8a0e62d57c45 admin admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:53:54.578123 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377b338d-8417-466a-a5e2-8a0e62d57c45 admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:53:54.578391 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377b338d-8417-466a-a5e2-8a0e62d57c45 admin admin] PATH_INFO: `/v3/roles` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:53:54.579143 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377b338d-8417-466a-a5e2-8a0e62d57c45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:53:54.580858 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377b338d-8417-466a-a5e2-8a0e62d57c45 admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:53:54.596658 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1296/2567] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 10:53:54 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 10:53:54.598138 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c67aad-2204-4164-bd05-8584a6fbf8b2 admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:53:54.609143 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c67aad-2204-4164-bd05-8584a6fbf8b2 admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:53:54.615815 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfe231e-bf6b-4921-8832-45ffbd64efa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:53:54.616508 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfe231e-bf6b-4921-8832-45ffbd64efa6 admin admin] REQUEST_METHOD: `PUT` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:53:54.616791 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfe231e-bf6b-4921-8832-45ffbd64efa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:53:54.617068 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfe231e-bf6b-4921-8832-45ffbd64efa6 admin admin] PATH_INFO: `/v3/projects/069d5edd9d9346bdb10c80922a763605/users/afd04d5fdc7e44c89edddf97be489db9/roles/4c9fc0b3baed441da2126be41cc6b04a` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:53:54.637721 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfe231e-bf6b-4921-8832-45ffbd64efa6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=069d5edd9d9346bdb10c80922a763605, user_id=afd04d5fdc7e44c89edddf97be489db9, role_id=4c9fc0b3baed441da2126be41cc6b04a)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:53:54.640548 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfe231e-bf6b-4921-8832-45ffbd64efa6 admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:53:54.661128 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1272/2568] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 10:53:54 2022] PUT /identity/v3/projects/069d5edd9d9346bdb10c80922a763605/users/afd04d5fdc7e44c89edddf97be489db9/roles/4c9fc0b3baed441da2126be41cc6b04a => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 10:53:54.662795 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33bf114-face-45fd-a17f-acdc2c0bbb38 None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:53:54.663103 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33bf114-face-45fd-a17f-acdc2c0bbb38 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:53:54.663382 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33bf114-face-45fd-a17f-acdc2c0bbb38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:53:54.685485 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-c33bf114-face-45fd-a17f-acdc2c0bbb38 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:53:54.687069 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-c33bf114-face-45fd-a17f-acdc2c0bbb38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:53:54.750878 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1297/2569] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:53:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:53:54.762130 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfe231e-bf6b-4921-8832-45ffbd64efa6 admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:53:54.763930 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-0dfe231e-bf6b-4921-8832-45ffbd64efa6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:53:54.830217 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfe231e-bf6b-4921-8832-45ffbd64efa6 admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:53:54.834336 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4d3320-2af4-491a-9044-85f1b60a8aef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7922d0a1a7c4e808d0b2a80dca0da40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:53:54.835010 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4d3320-2af4-491a-9044-85f1b60a8aef service neutron] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:53:54.837903 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4d3320-2af4-491a-9044-85f1b60a8aef service neutron] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:53:54.838180 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4d3320-2af4-491a-9044-85f1b60a8aef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:53:54.847913 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4d3320-2af4-491a-9044-85f1b60a8aef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:53:54.849523 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4d3320-2af4-491a-9044-85f1b60a8aef service neutron] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:53:54.864520 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1273/2570] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:53:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 10:54:01.655912 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ae4180-5784-467a-87de-b764850eb575 None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:54:01.656333 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ae4180-5784-467a-87de-b764850eb575 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:54:01.656524 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ae4180-5784-467a-87de-b764850eb575 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:54:01.699534 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-b1ae4180-5784-467a-87de-b764850eb575 None None] MFA Rules not processed for user `afd04d5fdc7e44c89edddf97be489db9`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:54:01.700875 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-b1ae4180-5784-467a-87de-b764850eb575 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:54:01.800807 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1298/2571] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:54:01 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:54:01.830158 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4d3320-2af4-491a-9044-85f1b60a8aef service neutron] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:54:01.834435 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-0b4d3320-2af4-491a-9044-85f1b60a8aef service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:54:01.917167 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=91,keystone:DELETE=6,keystone:INSERT=3 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:54:02.008091 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4d3320-2af4-491a-9044-85f1b60a8aef service neutron] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:54:02.034778 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c3615c-138f-4d24-a860-e48fd407db57 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e059d8b6c28c4fcba925d05d7c98e4ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:54:02.035442 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c3615c-138f-4d24-a860-e48fd407db57 service nova] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:54:02.036413 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c3615c-138f-4d24-a860-e48fd407db57 service nova] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:54:02.036704 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c3615c-138f-4d24-a860-e48fd407db57 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:54:02.064444 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c3615c-138f-4d24-a860-e48fd407db57 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:54:02.066078 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c3615c-138f-4d24-a860-e48fd407db57 service nova] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:54:02.109540 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1274/2572] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:54:01 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 275 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 10:54:03.651525 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0d06a9-ae76-4b33-83f6-8d52df662829 None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:54:03.652005 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0d06a9-ae76-4b33-83f6-8d52df662829 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:54:03.652101 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0d06a9-ae76-4b33-83f6-8d52df662829 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:54:03.675668 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-cf0d06a9-ae76-4b33-83f6-8d52df662829 None None] MFA Rules not processed for user `afd04d5fdc7e44c89edddf97be489db9`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:54:03.677164 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-cf0d06a9-ae76-4b33-83f6-8d52df662829 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:54:03.684959 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1299/2573] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:54:03 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:54:08.174901 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bd0318-1659-4d98-9ef3-d52cd74da29d None None] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:54:08.175403 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bd0318-1659-4d98-9ef3-d52cd74da29d None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:54:08.175650 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bd0318-1659-4d98-9ef3-d52cd74da29d None None] PATH_INFO: `/` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:54:08.178073 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1275/2574] 10.222.0.36 () {54 vars in 921 bytes} [Wed Jul 27 10:54:08 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 10:54:08.184842 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52b3316-d0be-4b99-9693-61a27e5c741b None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:54:08.185371 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52b3316-d0be-4b99-9693-61a27e5c741b None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:54:08.185371 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52b3316-d0be-4b99-9693-61a27e5c741b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:54:08.210909 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-f52b3316-d0be-4b99-9693-61a27e5c741b None None] MFA Rules not processed for user `e059d8b6c28c4fcba925d05d7c98e4ac`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:54:08.212526 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-f52b3316-d0be-4b99-9693-61a27e5c741b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:54:08.224170 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1300/2575] 10.222.0.36 () {58 vars in 1018 bytes} [Wed Jul 27 10:54:08 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 10:54:08.236938 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bd0318-1659-4d98-9ef3-d52cd74da29d None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:54:08.239291 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-06bd0318-1659-4d98-9ef3-d52cd74da29d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:54:08.314435 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bd0318-1659-4d98-9ef3-d52cd74da29d None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:54:08.330969 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb26ac15-aa04-499b-9347-42d38419fa6b service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc03c5d232654878aa1ddce15f49fdaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:54:08.330969 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb26ac15-aa04-499b-9347-42d38419fa6b service glance] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:54:08.330969 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb26ac15-aa04-499b-9347-42d38419fa6b service glance] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:54:08.330969 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb26ac15-aa04-499b-9347-42d38419fa6b service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:54:08.333465 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb26ac15-aa04-499b-9347-42d38419fa6b service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:54:08.335479 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb26ac15-aa04-499b-9347-42d38419fa6b service glance] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:54:08.348378 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1276/2576] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:54:08 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 10:54:08.425938 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21c6af8-5bb2-4870-986e-82cf282a540c None None] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:54:08.426236 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21c6af8-5bb2-4870-986e-82cf282a540c None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:54:08.426516 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21c6af8-5bb2-4870-986e-82cf282a540c None None] PATH_INFO: `/` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:54:08.427916 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1301/2577] 10.222.0.36 () {54 vars in 921 bytes} [Wed Jul 27 10:54:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 10:54:08.433239 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949e7470-fd60-42a2-b3f5-01cb74efd897 None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:54:08.433675 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949e7470-fd60-42a2-b3f5-01cb74efd897 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:54:08.434291 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949e7470-fd60-42a2-b3f5-01cb74efd897 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:54:08.459332 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-949e7470-fd60-42a2-b3f5-01cb74efd897 None None] MFA Rules not processed for user `c7922d0a1a7c4e808d0b2a80dca0da40`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:54:08.460750 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-949e7470-fd60-42a2-b3f5-01cb74efd897 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:54:08.469177 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1277/2578] 10.222.0.36 () {58 vars in 1018 bytes} [Wed Jul 27 10:54:08 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:54:08.489136 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21c6af8-5bb2-4870-986e-82cf282a540c None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:54:08.491173 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-d21c6af8-5bb2-4870-986e-82cf282a540c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:54:08.505264 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21c6af8-5bb2-4870-986e-82cf282a540c None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:54:08.514654 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2687c1bc-1bac-4fc1-9cd2-17cd817c17a9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7922d0a1a7c4e808d0b2a80dca0da40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:54:08.515419 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2687c1bc-1bac-4fc1-9cd2-17cd817c17a9 service neutron] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:54:08.515773 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2687c1bc-1bac-4fc1-9cd2-17cd817c17a9 service neutron] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:54:08.516506 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2687c1bc-1bac-4fc1-9cd2-17cd817c17a9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:54:08.522515 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:54:08.525650 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2687c1bc-1bac-4fc1-9cd2-17cd817c17a9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:54:08.527804 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2687c1bc-1bac-4fc1-9cd2-17cd817c17a9 service neutron] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:54:08.537644 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1302/2579] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:54:08 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 10:54:08.665855 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae222926-9eca-488b-befb-b5f4286fbacd None None] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:54:08.666146 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae222926-9eca-488b-befb-b5f4286fbacd None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:54:08.666425 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae222926-9eca-488b-befb-b5f4286fbacd None None] PATH_INFO: `/` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:54:08.667873 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1278/2580] 10.222.0.36 () {54 vars in 921 bytes} [Wed Jul 27 10:54:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 10:54:08.673942 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec12166-0a34-4776-988f-e915f837acad None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:54:08.674303 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec12166-0a34-4776-988f-e915f837acad None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:54:08.674671 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec12166-0a34-4776-988f-e915f837acad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:54:08.697521 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-eec12166-0a34-4776-988f-e915f837acad None None] MFA Rules not processed for user `e059d8b6c28c4fcba925d05d7c98e4ac`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:54:08.699313 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-eec12166-0a34-4776-988f-e915f837acad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:54:08.711655 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1303/2581] 10.222.0.36 () {58 vars in 1018 bytes} [Wed Jul 27 10:54:08 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 10:54:08.723409 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae222926-9eca-488b-befb-b5f4286fbacd None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:54:08.735946 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae222926-9eca-488b-befb-b5f4286fbacd None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:54:08.740205 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9e86a6-2bd5-49a6-b346-6a15e61055fe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7922d0a1a7c4e808d0b2a80dca0da40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:54:08.740839 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9e86a6-2bd5-49a6-b346-6a15e61055fe service neutron] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:54:08.741121 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9e86a6-2bd5-49a6-b346-6a15e61055fe service neutron] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:54:08.741395 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9e86a6-2bd5-49a6-b346-6a15e61055fe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:54:08.753632 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9e86a6-2bd5-49a6-b346-6a15e61055fe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:54:08.755370 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9e86a6-2bd5-49a6-b346-6a15e61055fe service neutron] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:54:08.768713 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1279/2582] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:54:08 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 10:54:11.384925 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bfaca5-0988-4ce2-bd9b-575f78792c53 None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:54:11.385672 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bfaca5-0988-4ce2-bd9b-575f78792c53 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:54:11.385961 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bfaca5-0988-4ce2-bd9b-575f78792c53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:54:11.405609 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-e2bfaca5-0988-4ce2-bd9b-575f78792c53 None None] MFA Rules not processed for user `3d9fed24b4ba44b0b5961a603c4362c4`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:54:11.406849 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-e2bfaca5-0988-4ce2-bd9b-575f78792c53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:54:11.474784 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1304/2583] 10.222.0.36 () {58 vars in 1009 bytes} [Wed Jul 27 10:54:11 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:54:11.488322 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9e86a6-2bd5-49a6-b346-6a15e61055fe service neutron] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:54:11.489892 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-ac9e86a6-2bd5-49a6-b346-6a15e61055fe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:54:11.573503 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9e86a6-2bd5-49a6-b346-6a15e61055fe service neutron] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:54:11.579010 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea90740-a9d2-4a15-aa10-aa4c88f980a4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89ac506eefcd4e42a84b44fa15fb3975', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:54:11.580090 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea90740-a9d2-4a15-aa10-aa4c88f980a4 service swift] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:54:11.580571 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea90740-a9d2-4a15-aa10-aa4c88f980a4 service swift] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:54:11.580854 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea90740-a9d2-4a15-aa10-aa4c88f980a4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:54:11.591340 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea90740-a9d2-4a15-aa10-aa4c88f980a4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:54:11.592927 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea90740-a9d2-4a15-aa10-aa4c88f980a4 service swift] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:54:11.601471 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1280/2584] 10.222.0.36 () {60 vars in 1384 bytes} [Wed Jul 27 10:54:11 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 27 10:54:21.464595 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=30 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:54:21.597743 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=78 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:54:24.450086 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bfaca5-0988-4ce2-bd9b-575f78792c53 None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:54:24.464508 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bfaca5-0988-4ce2-bd9b-575f78792c53 None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:54:24.469954 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6311b733-1905-405c-847d-614fd61c0beb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e059d8b6c28c4fcba925d05d7c98e4ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:54:24.470580 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6311b733-1905-405c-847d-614fd61c0beb service nova] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:54:24.470900 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6311b733-1905-405c-847d-614fd61c0beb service nova] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:54:24.471180 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6311b733-1905-405c-847d-614fd61c0beb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:54:24.472971 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-6311b733-1905-405c-847d-614fd61c0beb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:54:24.485633 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6311b733-1905-405c-847d-614fd61c0beb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:54:24.487295 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6311b733-1905-405c-847d-614fd61c0beb service nova] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:54:24.498874 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1305/2585] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:54:24 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 10:54:26.710489 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28255735-7fab-47d2-b1db-ae978fdf184b None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:54:26.710987 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28255735-7fab-47d2-b1db-ae978fdf184b None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:54:26.711076 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28255735-7fab-47d2-b1db-ae978fdf184b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:54:26.731195 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-28255735-7fab-47d2-b1db-ae978fdf184b None None] MFA Rules not processed for user `3d9fed24b4ba44b0b5961a603c4362c4`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:54:26.732401 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-28255735-7fab-47d2-b1db-ae978fdf184b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:54:26.739544 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1281/2586] 10.222.0.36 () {58 vars in 967 bytes} [Wed Jul 27 10:54:26 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:54:26.751675 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6311b733-1905-405c-847d-614fd61c0beb service nova] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:54:26.762743 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6311b733-1905-405c-847d-614fd61c0beb service nova] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:54:26.767696 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea986401-447e-47f4-847a-69298d80d1e7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89ac506eefcd4e42a84b44fa15fb3975', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:54:26.768299 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea986401-447e-47f4-847a-69298d80d1e7 service swift] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:54:26.768578 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea986401-447e-47f4-847a-69298d80d1e7 service swift] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:54:26.768849 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea986401-447e-47f4-847a-69298d80d1e7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:54:26.777245 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea986401-447e-47f4-847a-69298d80d1e7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:54:26.778829 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea986401-447e-47f4-847a-69298d80d1e7 service swift] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:54:26.786732 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1306/2587] 10.222.0.36 () {60 vars in 1384 bytes} [Wed Jul 27 10:54:26 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 27 10:54:36.725229 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=3 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:54:36.783538 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=12 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:54:54.774469 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837c09cb-7f63-4a20-afc5-525815b50fe5 None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:54:54.775110 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837c09cb-7f63-4a20-afc5-525815b50fe5 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:54:54.775219 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837c09cb-7f63-4a20-afc5-525815b50fe5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:54:54.796920 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-837c09cb-7f63-4a20-afc5-525815b50fe5 None None] MFA Rules not processed for user `0621626f2863400c903de6a88da84f1e`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:54:54.798104 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-837c09cb-7f63-4a20-afc5-525815b50fe5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:54:54.852317 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1282/2588] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:54:54 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:54:55.081335 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea986401-447e-47f4-847a-69298d80d1e7 service swift] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:54:55.090634 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea986401-447e-47f4-847a-69298d80d1e7 service swift] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:54:55.094216 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c4f351-adde-4fec-aff0-225400cace3b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7922d0a1a7c4e808d0b2a80dca0da40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:54:55.094859 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c4f351-adde-4fec-aff0-225400cace3b service neutron] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:54:55.095147 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c4f351-adde-4fec-aff0-225400cace3b service neutron] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:54:55.095427 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c4f351-adde-4fec-aff0-225400cace3b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:54:55.097140 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-70c4f351-adde-4fec-aff0-225400cace3b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:54:55.106999 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c4f351-adde-4fec-aff0-225400cace3b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:54:55.108413 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c4f351-adde-4fec-aff0-225400cace3b service neutron] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:54:55.116296 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1307/2589] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:54:55 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 10:55:04.845569 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=25 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:55:05.112842 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=6 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:55:06.959873 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837c09cb-7f63-4a20-afc5-525815b50fe5 None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:55:06.961286 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-837c09cb-7f63-4a20-afc5-525815b50fe5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:55:07.050894 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837c09cb-7f63-4a20-afc5-525815b50fe5 None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:55:07.059830 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e1c5a3-bcd6-473c-af4d-17bc3e11a54a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '864d35f268cf45148756109759bb2743', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:55:07.060446 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e1c5a3-bcd6-473c-af4d-17bc3e11a54a service cinder] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:55:07.060726 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e1c5a3-bcd6-473c-af4d-17bc3e11a54a service cinder] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:55:07.060998 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e1c5a3-bcd6-473c-af4d-17bc3e11a54a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:55:07.062850 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-a6e1c5a3-bcd6-473c-af4d-17bc3e11a54a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:55:07.121214 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e1c5a3-bcd6-473c-af4d-17bc3e11a54a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:55:07.122749 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e1c5a3-bcd6-473c-af4d-17bc3e11a54a service cinder] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:55:07.131724 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1283/2590] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:55:06 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 10:55:17.128187 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=48 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:55:17.194954 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c4f351-adde-4fec-aff0-225400cace3b service neutron] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:55:17.204361 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c4f351-adde-4fec-aff0-225400cace3b service neutron] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:55:17.208060 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a360e53-4a64-47ce-b35d-2eb9904790ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7922d0a1a7c4e808d0b2a80dca0da40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:55:17.208693 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a360e53-4a64-47ce-b35d-2eb9904790ba service neutron] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:55:17.208979 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a360e53-4a64-47ce-b35d-2eb9904790ba service neutron] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:55:17.209256 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a360e53-4a64-47ce-b35d-2eb9904790ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:55:17.211006 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-2a360e53-4a64-47ce-b35d-2eb9904790ba service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:55:17.222851 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a360e53-4a64-47ce-b35d-2eb9904790ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:55:17.224275 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a360e53-4a64-47ce-b35d-2eb9904790ba service neutron] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:55:17.233767 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1308/2591] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:55:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 10:55:20.311009 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e1c5a3-bcd6-473c-af4d-17bc3e11a54a service cinder] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:55:20.312708 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-a6e1c5a3-bcd6-473c-af4d-17bc3e11a54a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:55:20.341511 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e1c5a3-bcd6-473c-af4d-17bc3e11a54a service cinder] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:55:20.346375 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e87835-671a-464b-b17f-302e6b94e8f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:55:20.347085 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e87835-671a-464b-b17f-302e6b94e8f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:55:20.347381 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e87835-671a-464b-b17f-302e6b94e8f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:55:20.347663 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e87835-671a-464b-b17f-302e6b94e8f7 admin admin] PATH_INFO: `/v3/users/fc1270b519b44320a772d7ad163c873d` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:55:20.348872 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e87835-671a-464b-b17f-302e6b94e8f7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fc1270b519b44320a772d7ad163c873d)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:55:20.350970 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e87835-671a-464b-b17f-302e6b94e8f7 admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:55:20.404065 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-26e87835-671a-464b-b17f-302e6b94e8f7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fc1270b519b44320a772d7ad163c873d'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:55:20.411365 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-26e87835-671a-464b-b17f-302e6b94e8f7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fc1270b519b44320a772d7ad163c873d'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:55:20.418664 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-26e87835-671a-464b-b17f-302e6b94e8f7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fc1270b519b44320a772d7ad163c873d'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:55:20.419361 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-26e87835-671a-464b-b17f-302e6b94e8f7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fc1270b519b44320a772d7ad163c873d'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:55:20.419998 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-26e87835-671a-464b-b17f-302e6b94e8f7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fc1270b519b44320a772d7ad163c873d'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:55:20.437188 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1284/2592] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Jul 27 10:55:20 2022] DELETE /identity/v3/users/fc1270b519b44320a772d7ad163c873d => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:55:20.611577 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a360e53-4a64-47ce-b35d-2eb9904790ba service neutron] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:55:20.622797 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a360e53-4a64-47ce-b35d-2eb9904790ba service neutron] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:55:20.678906 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198c5fe1-8e76-4d5f-a752-e4df166f8d02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:55:20.679557 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198c5fe1-8e76-4d5f-a752-e4df166f8d02 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:55:20.679837 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198c5fe1-8e76-4d5f-a752-e4df166f8d02 admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:55:20.680112 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198c5fe1-8e76-4d5f-a752-e4df166f8d02 admin admin] PATH_INFO: `/v3/projects/1f86323e8542471190e0b58769416138` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:55:20.681917 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198c5fe1-8e76-4d5f-a752-e4df166f8d02 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1f86323e8542471190e0b58769416138)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:55:20.683860 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198c5fe1-8e76-4d5f-a752-e4df166f8d02 admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:55:20.731003 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-198c5fe1-8e76-4d5f-a752-e4df166f8d02 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1f86323e8542471190e0b58769416138', 'request_id': 'req-198c5fe1-8e76-4d5f-a752-e4df166f8d02'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:55:20.737786 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-198c5fe1-8e76-4d5f-a752-e4df166f8d02 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1f86323e8542471190e0b58769416138', 'request_id': 'req-198c5fe1-8e76-4d5f-a752-e4df166f8d02'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:55:20.744833 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-198c5fe1-8e76-4d5f-a752-e4df166f8d02 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1f86323e8542471190e0b58769416138', 'request_id': 'req-198c5fe1-8e76-4d5f-a752-e4df166f8d02'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:55:20.755266 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-198c5fe1-8e76-4d5f-a752-e4df166f8d02 admin admin] The token cache is being invalidate because project 1f86323e8542471190e0b58769416138 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86827) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 10:55:20.755553 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-198c5fe1-8e76-4d5f-a752-e4df166f8d02 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:55:20.756170 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-198c5fe1-8e76-4d5f-a752-e4df166f8d02 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:55:20.757673 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1309/2593] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Jul 27 10:55:20 2022] DELETE /identity/v3/projects/1f86323e8542471190e0b58769416138 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:55:20.762618 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cd7322-4763-46e9-83ed-62cccd810a58 None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:55:20.762913 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cd7322-4763-46e9-83ed-62cccd810a58 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:55:20.763192 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cd7322-4763-46e9-83ed-62cccd810a58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:55:20.782757 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-67cd7322-4763-46e9-83ed-62cccd810a58 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:55:20.783961 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-67cd7322-4763-46e9-83ed-62cccd810a58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:55:20.842428 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1285/2594] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:55:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:55:20.850651 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542186a9-f1bc-4c78-86c6-10351a263582 None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:55:20.850952 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542186a9-f1bc-4c78-86c6-10351a263582 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:55:20.851250 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542186a9-f1bc-4c78-86c6-10351a263582 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:55:20.870957 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-542186a9-f1bc-4c78-86c6-10351a263582 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:55:20.872161 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-542186a9-f1bc-4c78-86c6-10351a263582 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:55:20.880262 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1310/2595] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:55:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:55:20.884397 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cd7322-4763-46e9-83ed-62cccd810a58 None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:55:20.894172 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cd7322-4763-46e9-83ed-62cccd810a58 None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:55:20.898854 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec8c5a8-c610-469b-8cd9-382d4ef81067 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:55:20.899465 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec8c5a8-c610-469b-8cd9-382d4ef81067 admin admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:55:20.899748 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec8c5a8-c610-469b-8cd9-382d4ef81067 admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:55:20.900596 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec8c5a8-c610-469b-8cd9-382d4ef81067 admin admin] PATH_INFO: `/v3/domains` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:55:20.901315 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec8c5a8-c610-469b-8cd9-382d4ef81067 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86828) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:55:20.901625 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec8c5a8-c610-469b-8cd9-382d4ef81067 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:55:20.903260 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec8c5a8-c610-469b-8cd9-382d4ef81067 admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:55:20.914778 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1286/2596] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Jul 27 10:55:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 10:55:20.919709 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542186a9-f1bc-4c78-86c6-10351a263582 None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:55:20.928954 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542186a9-f1bc-4c78-86c6-10351a263582 None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:55:20.933684 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfbd891-b696-44e1-afd0-1569f6ba660a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:55:20.934287 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfbd891-b696-44e1-afd0-1569f6ba660a admin admin] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:55:20.934573 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfbd891-b696-44e1-afd0-1569f6ba660a admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:55:20.934885 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfbd891-b696-44e1-afd0-1569f6ba660a admin admin] PATH_INFO: `/v3/projects` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:55:20.935547 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abfbd891-b696-44e1-afd0-1569f6ba660a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:55:20.937484 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abfbd891-b696-44e1-afd0-1569f6ba660a admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:55:20.965268 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1311/2597] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Jul 27 10:55:20 2022] POST /identity/v3/projects => generated 367 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 10:55:20.969351 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec8c5a8-c610-469b-8cd9-382d4ef81067 admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:55:20.980166 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec8c5a8-c610-469b-8cd9-382d4ef81067 admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:55:21.036681 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab858852-7822-4e66-b6a4-f55add3b676f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:55:21.037231 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab858852-7822-4e66-b6a4-f55add3b676f admin admin] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:55:21.037544 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab858852-7822-4e66-b6a4-f55add3b676f admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:55:21.037828 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab858852-7822-4e66-b6a4-f55add3b676f admin admin] PATH_INFO: `/v3/users` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:55:21.038504 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab858852-7822-4e66-b6a4-f55add3b676f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:55:21.040507 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab858852-7822-4e66-b6a4-f55add3b676f admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:55:21.048792 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:55:21.063512 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1287/2598] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Jul 27 10:55:20 2022] POST /identity/v3/users => generated 284 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 10:55:21.067817 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfbd891-b696-44e1-afd0-1569f6ba660a admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:55:21.078553 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfbd891-b696-44e1-afd0-1569f6ba660a admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:55:21.083145 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c41f421-31d5-4494-aec0-f08427f21d4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:55:21.083754 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c41f421-31d5-4494-aec0-f08427f21d4a admin admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:55:21.084038 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c41f421-31d5-4494-aec0-f08427f21d4a admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:55:21.084316 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c41f421-31d5-4494-aec0-f08427f21d4a admin admin] PATH_INFO: `/v3/roles` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:55:21.085029 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c41f421-31d5-4494-aec0-f08427f21d4a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:55:21.086624 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c41f421-31d5-4494-aec0-f08427f21d4a admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:55:21.096929 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1312/2599] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 10:55:21 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 10:55:21.100574 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab858852-7822-4e66-b6a4-f55add3b676f admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:55:21.110797 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab858852-7822-4e66-b6a4-f55add3b676f admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:55:21.115601 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757d183f-5c66-4f30-9c59-7417e716a5ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:55:21.116244 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757d183f-5c66-4f30-9c59-7417e716a5ec admin admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:55:21.116571 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757d183f-5c66-4f30-9c59-7417e716a5ec admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:55:21.116800 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757d183f-5c66-4f30-9c59-7417e716a5ec admin admin] PATH_INFO: `/v3/roles` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:55:21.117495 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757d183f-5c66-4f30-9c59-7417e716a5ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:55:21.119141 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757d183f-5c66-4f30-9c59-7417e716a5ec admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:55:21.131491 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1288/2600] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 10:55:21 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 10:55:21.135969 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c41f421-31d5-4494-aec0-f08427f21d4a admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:55:21.154332 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c41f421-31d5-4494-aec0-f08427f21d4a admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:55:21.159210 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f64000-8412-4a5a-b407-745556aec4a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:55:21.159885 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f64000-8412-4a5a-b407-745556aec4a1 admin admin] REQUEST_METHOD: `PUT` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:55:21.160172 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f64000-8412-4a5a-b407-745556aec4a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:55:21.160455 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f64000-8412-4a5a-b407-745556aec4a1 admin admin] PATH_INFO: `/v3/projects/5ae3bbdf81794b1ab67077fa788f0092/users/1c4ceb586908425883372c16985f5cc4/roles/4c9fc0b3baed441da2126be41cc6b04a` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:55:21.175557 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f64000-8412-4a5a-b407-745556aec4a1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ae3bbdf81794b1ab67077fa788f0092, user_id=1c4ceb586908425883372c16985f5cc4, role_id=4c9fc0b3baed441da2126be41cc6b04a)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:55:21.178200 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f64000-8412-4a5a-b407-745556aec4a1 admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:55:21.191471 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1313/2601] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 10:55:21 2022] PUT /identity/v3/projects/5ae3bbdf81794b1ab67077fa788f0092/users/1c4ceb586908425883372c16985f5cc4/roles/4c9fc0b3baed441da2126be41cc6b04a => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:55:21.203265 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fc4ce5-8097-4e38-853a-21c9b7039eb3 None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:55:21.203558 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fc4ce5-8097-4e38-853a-21c9b7039eb3 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:55:21.203835 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fc4ce5-8097-4e38-853a-21c9b7039eb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:55:21.232115 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-22fc4ce5-8097-4e38-853a-21c9b7039eb3 None None] MFA Rules not processed for user `1c4ceb586908425883372c16985f5cc4`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:55:21.233303 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-22fc4ce5-8097-4e38-853a-21c9b7039eb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:55:21.296015 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1289/2602] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:55:21 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:55:21.307248 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f64000-8412-4a5a-b407-745556aec4a1 admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:55:21.308945 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-f2f64000-8412-4a5a-b407-745556aec4a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:55:21.323358 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:55:21.367010 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f64000-8412-4a5a-b407-745556aec4a1 admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:55:21.371322 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50bbb65-d38c-416a-acbe-f6ad58a76350 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '864d35f268cf45148756109759bb2743', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:55:21.371956 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50bbb65-d38c-416a-acbe-f6ad58a76350 service cinder] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:55:21.372225 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50bbb65-d38c-416a-acbe-f6ad58a76350 service cinder] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:55:21.372501 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50bbb65-d38c-416a-acbe-f6ad58a76350 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:55:21.381605 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50bbb65-d38c-416a-acbe-f6ad58a76350 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:55:21.383120 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50bbb65-d38c-416a-acbe-f6ad58a76350 service cinder] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:55:21.392228 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1314/2603] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:55:21 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 10:55:23.573235 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf24cee4-a931-4e1d-86e4-63e64e0be212 None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:55:23.573769 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf24cee4-a931-4e1d-86e4-63e64e0be212 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:55:23.573869 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf24cee4-a931-4e1d-86e4-63e64e0be212 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:55:23.593354 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-bf24cee4-a931-4e1d-86e4-63e64e0be212 None None] MFA Rules not processed for user `afd04d5fdc7e44c89edddf97be489db9`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:55:23.594642 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-bf24cee4-a931-4e1d-86e4-63e64e0be212 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:55:23.651786 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1290/2604] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:55:23 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:55:30.123752 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50bbb65-d38c-416a-acbe-f6ad58a76350 service cinder] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:55:30.225652 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50bbb65-d38c-416a-acbe-f6ad58a76350 service cinder] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:55:30.231670 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3230043-19e3-4f35-8281-7e491be81d6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:55:30.232334 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3230043-19e3-4f35-8281-7e491be81d6d admin admin] REQUEST_METHOD: `DELETE` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:55:30.232684 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3230043-19e3-4f35-8281-7e491be81d6d admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:55:30.233037 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3230043-19e3-4f35-8281-7e491be81d6d admin admin] PATH_INFO: `/v3/users/1c4ceb586908425883372c16985f5cc4` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:55:30.234108 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3230043-19e3-4f35-8281-7e491be81d6d admin admin] RBAC: Authorizing `identity:delete_user(user_id=1c4ceb586908425883372c16985f5cc4)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:55:30.236136 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3230043-19e3-4f35-8281-7e491be81d6d admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:55:30.286526 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-e3230043-19e3-4f35-8281-7e491be81d6d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1c4ceb586908425883372c16985f5cc4'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:55:30.293871 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-e3230043-19e3-4f35-8281-7e491be81d6d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1c4ceb586908425883372c16985f5cc4'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:55:30.309818 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-e3230043-19e3-4f35-8281-7e491be81d6d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1c4ceb586908425883372c16985f5cc4'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:55:30.310516 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-e3230043-19e3-4f35-8281-7e491be81d6d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1c4ceb586908425883372c16985f5cc4'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:55:30.311261 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-e3230043-19e3-4f35-8281-7e491be81d6d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1c4ceb586908425883372c16985f5cc4'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:55:30.330710 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1315/2605] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Jul 27 10:55:30 2022] DELETE /identity/v3/users/1c4ceb586908425883372c16985f5cc4 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:55:30.335120 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640f1b2a-ce3f-4968-8e9a-d553a41079ee None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:55:30.335588 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640f1b2a-ce3f-4968-8e9a-d553a41079ee None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:55:30.335813 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640f1b2a-ce3f-4968-8e9a-d553a41079ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:55:30.355071 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-640f1b2a-ce3f-4968-8e9a-d553a41079ee None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:55:30.356415 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-640f1b2a-ce3f-4968-8e9a-d553a41079ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:55:30.419021 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1291/2606] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:55:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:55:30.428747 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3230043-19e3-4f35-8281-7e491be81d6d admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:55:30.430139 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-e3230043-19e3-4f35-8281-7e491be81d6d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:55:30.471701 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:55:30.491020 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3230043-19e3-4f35-8281-7e491be81d6d admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:55:30.494739 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e787d3d5-5f21-4a30-880a-0a147bc720a9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7922d0a1a7c4e808d0b2a80dca0da40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:55:30.495365 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e787d3d5-5f21-4a30-880a-0a147bc720a9 service neutron] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:55:30.495652 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e787d3d5-5f21-4a30-880a-0a147bc720a9 service neutron] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:55:30.495932 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e787d3d5-5f21-4a30-880a-0a147bc720a9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:55:30.505199 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e787d3d5-5f21-4a30-880a-0a147bc720a9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:55:30.506657 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e787d3d5-5f21-4a30-880a-0a147bc720a9 service neutron] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:55:30.516124 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1316/2607] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:55:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 10:55:30.780460 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640f1b2a-ce3f-4968-8e9a-d553a41079ee None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:55:30.781896 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-640f1b2a-ce3f-4968-8e9a-d553a41079ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:55:30.794496 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640f1b2a-ce3f-4968-8e9a-d553a41079ee None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:55:30.799167 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e93755-6d91-42b2-b3eb-734c72101c77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:55:30.799829 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e93755-6d91-42b2-b3eb-734c72101c77 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:55:30.800113 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e93755-6d91-42b2-b3eb-734c72101c77 admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:55:30.800400 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e93755-6d91-42b2-b3eb-734c72101c77 admin admin] PATH_INFO: `/v3/projects/5ae3bbdf81794b1ab67077fa788f0092` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:55:30.801598 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e93755-6d91-42b2-b3eb-734c72101c77 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5ae3bbdf81794b1ab67077fa788f0092)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:55:30.803655 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e93755-6d91-42b2-b3eb-734c72101c77 admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:55:30.812152 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:55:30.853784 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-54e93755-6d91-42b2-b3eb-734c72101c77 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5ae3bbdf81794b1ab67077fa788f0092', 'request_id': 'req-54e93755-6d91-42b2-b3eb-734c72101c77'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:55:30.859930 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-54e93755-6d91-42b2-b3eb-734c72101c77 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5ae3bbdf81794b1ab67077fa788f0092', 'request_id': 'req-54e93755-6d91-42b2-b3eb-734c72101c77'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:55:30.867188 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-54e93755-6d91-42b2-b3eb-734c72101c77 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5ae3bbdf81794b1ab67077fa788f0092', 'request_id': 'req-54e93755-6d91-42b2-b3eb-734c72101c77'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:55:30.877754 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-54e93755-6d91-42b2-b3eb-734c72101c77 admin admin] The token cache is being invalidate because project 5ae3bbdf81794b1ab67077fa788f0092 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86828) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 10:55:30.878042 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-54e93755-6d91-42b2-b3eb-734c72101c77 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:55:30.878703 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-54e93755-6d91-42b2-b3eb-734c72101c77 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:55:30.880205 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1292/2608] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Jul 27 10:55:30 2022] DELETE /identity/v3/projects/5ae3bbdf81794b1ab67077fa788f0092 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:55:30.885684 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e91f8ae-091e-4cf4-b1bc-167a461348c6 None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:55:30.885981 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e91f8ae-091e-4cf4-b1bc-167a461348c6 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:55:30.886294 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e91f8ae-091e-4cf4-b1bc-167a461348c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:55:30.905965 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-5e91f8ae-091e-4cf4-b1bc-167a461348c6 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:55:30.907197 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-5e91f8ae-091e-4cf4-b1bc-167a461348c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:55:30.965769 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1317/2609] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:55:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:55:30.973815 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6e9208-7d89-4c8b-9262-916e66c8b318 None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:55:30.974104 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6e9208-7d89-4c8b-9262-916e66c8b318 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:55:30.974381 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6e9208-7d89-4c8b-9262-916e66c8b318 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:55:30.993672 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-bd6e9208-7d89-4c8b-9262-916e66c8b318 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:55:30.994899 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-bd6e9208-7d89-4c8b-9262-916e66c8b318 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:55:31.003193 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1293/2610] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:55:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:55:31.007260 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e91f8ae-091e-4cf4-b1bc-167a461348c6 None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:55:31.017367 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e91f8ae-091e-4cf4-b1bc-167a461348c6 None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:55:31.021971 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460e36e0-a4af-4b94-b575-134c4e3c6d9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:55:31.022579 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460e36e0-a4af-4b94-b575-134c4e3c6d9c admin admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:55:31.022897 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460e36e0-a4af-4b94-b575-134c4e3c6d9c admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:55:31.023177 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460e36e0-a4af-4b94-b575-134c4e3c6d9c admin admin] PATH_INFO: `/v3/domains` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:55:31.023880 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460e36e0-a4af-4b94-b575-134c4e3c6d9c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86827) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:55:31.024186 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460e36e0-a4af-4b94-b575-134c4e3c6d9c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:55:31.026390 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460e36e0-a4af-4b94-b575-134c4e3c6d9c admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:55:31.037733 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1318/2611] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Jul 27 10:55:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:55:31.041798 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6e9208-7d89-4c8b-9262-916e66c8b318 None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:55:31.050806 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6e9208-7d89-4c8b-9262-916e66c8b318 None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:55:31.055466 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a174ec63-c7f1-476c-9fea-009338efdced admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:55:31.056066 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a174ec63-c7f1-476c-9fea-009338efdced admin admin] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:55:31.056350 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a174ec63-c7f1-476c-9fea-009338efdced admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:55:31.056627 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a174ec63-c7f1-476c-9fea-009338efdced admin admin] PATH_INFO: `/v3/projects` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:55:31.057277 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a174ec63-c7f1-476c-9fea-009338efdced admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:55:31.059229 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a174ec63-c7f1-476c-9fea-009338efdced admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:55:31.085989 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1294/2612] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Jul 27 10:55:31 2022] POST /identity/v3/projects => generated 363 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 10:55:31.090770 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460e36e0-a4af-4b94-b575-134c4e3c6d9c admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:55:31.100643 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460e36e0-a4af-4b94-b575-134c4e3c6d9c admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:55:31.156602 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6f7d51-c00a-48c5-a5e9-9bbc57afe514 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:55:31.157221 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6f7d51-c00a-48c5-a5e9-9bbc57afe514 admin admin] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:55:31.157501 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6f7d51-c00a-48c5-a5e9-9bbc57afe514 admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:55:31.157821 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6f7d51-c00a-48c5-a5e9-9bbc57afe514 admin admin] PATH_INFO: `/v3/users` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:55:31.158493 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6f7d51-c00a-48c5-a5e9-9bbc57afe514 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:55:31.160499 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6f7d51-c00a-48c5-a5e9-9bbc57afe514 admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:55:31.182787 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1319/2613] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Jul 27 10:55:31 2022] POST /identity/v3/users => generated 282 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 10:55:31.186705 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a174ec63-c7f1-476c-9fea-009338efdced admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:55:31.197953 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a174ec63-c7f1-476c-9fea-009338efdced admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:55:31.202858 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cfc4fe-249e-4de2-9f89-d2f3dfc1a6b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:55:31.203493 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cfc4fe-249e-4de2-9f89-d2f3dfc1a6b9 admin admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:55:31.203779 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cfc4fe-249e-4de2-9f89-d2f3dfc1a6b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:55:31.204060 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cfc4fe-249e-4de2-9f89-d2f3dfc1a6b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:55:31.204756 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33cfc4fe-249e-4de2-9f89-d2f3dfc1a6b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:55:31.206359 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33cfc4fe-249e-4de2-9f89-d2f3dfc1a6b9 admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:55:31.216728 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1295/2614] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 10:55:31 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 10:55:31.220872 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6f7d51-c00a-48c5-a5e9-9bbc57afe514 admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:55:31.231994 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6f7d51-c00a-48c5-a5e9-9bbc57afe514 admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:55:31.236892 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ada0055-55c3-4c44-bf4a-7ef63e2d9856 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:55:31.237548 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ada0055-55c3-4c44-bf4a-7ef63e2d9856 admin admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:55:31.237831 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ada0055-55c3-4c44-bf4a-7ef63e2d9856 admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:55:31.238243 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ada0055-55c3-4c44-bf4a-7ef63e2d9856 admin admin] PATH_INFO: `/v3/roles` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:55:31.238978 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ada0055-55c3-4c44-bf4a-7ef63e2d9856 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:55:31.240622 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ada0055-55c3-4c44-bf4a-7ef63e2d9856 admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:55:31.259887 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1320/2615] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 10:55:31 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 10:55:31.264051 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cfc4fe-249e-4de2-9f89-d2f3dfc1a6b9 admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:55:31.274075 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cfc4fe-249e-4de2-9f89-d2f3dfc1a6b9 admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:55:31.278918 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133b9c82-4131-4b2f-b7aa-f631988acb6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:55:31.279659 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133b9c82-4131-4b2f-b7aa-f631988acb6b admin admin] REQUEST_METHOD: `PUT` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:55:31.280013 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133b9c82-4131-4b2f-b7aa-f631988acb6b admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:55:31.280417 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133b9c82-4131-4b2f-b7aa-f631988acb6b admin admin] PATH_INFO: `/v3/projects/367fba0042ce43c79cdaaac707c07f2f/users/1b8cc040da10482e936e018ef2a5e194/roles/4c9fc0b3baed441da2126be41cc6b04a` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:55:31.295641 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133b9c82-4131-4b2f-b7aa-f631988acb6b admin admin] RBAC: Authorizing `identity:create_grant(project_id=367fba0042ce43c79cdaaac707c07f2f, user_id=1b8cc040da10482e936e018ef2a5e194, role_id=4c9fc0b3baed441da2126be41cc6b04a)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:55:31.298409 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133b9c82-4131-4b2f-b7aa-f631988acb6b admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:55:31.312371 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1296/2616] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 10:55:31 2022] PUT /identity/v3/projects/367fba0042ce43c79cdaaac707c07f2f/users/1b8cc040da10482e936e018ef2a5e194/roles/4c9fc0b3baed441da2126be41cc6b04a => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:55:31.322919 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df118776-1357-4892-b1e4-10cb97e657ed None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:55:31.323214 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df118776-1357-4892-b1e4-10cb97e657ed None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:55:31.323497 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df118776-1357-4892-b1e4-10cb97e657ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:55:31.350855 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-df118776-1357-4892-b1e4-10cb97e657ed None None] MFA Rules not processed for user `1b8cc040da10482e936e018ef2a5e194`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:55:31.352120 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-df118776-1357-4892-b1e4-10cb97e657ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:55:31.384162 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:55:31.418052 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1321/2617] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:55:31 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:55:31.422446 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133b9c82-4131-4b2f-b7aa-f631988acb6b admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:55:31.483010 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133b9c82-4131-4b2f-b7aa-f631988acb6b admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:55:31.487674 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5062bdb-9ab4-4056-a5d9-4da0d64bde54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:55:31.488293 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5062bdb-9ab4-4056-a5d9-4da0d64bde54 admin admin] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:55:31.488574 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5062bdb-9ab4-4056-a5d9-4da0d64bde54 admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:55:31.488845 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5062bdb-9ab4-4056-a5d9-4da0d64bde54 admin admin] PATH_INFO: `/v3/projects` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:55:31.489508 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5062bdb-9ab4-4056-a5d9-4da0d64bde54 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:55:31.491435 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5062bdb-9ab4-4056-a5d9-4da0d64bde54 admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:55:31.523664 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1297/2618] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Jul 27 10:55:31 2022] POST /identity/v3/projects => generated 363 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 10:55:31.525239 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df118776-1357-4892-b1e4-10cb97e657ed None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:55:31.535883 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df118776-1357-4892-b1e4-10cb97e657ed None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:55:31.591791 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d74b260-1883-4732-96d6-93c8b0e8cc2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:55:31.592406 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d74b260-1883-4732-96d6-93c8b0e8cc2a admin admin] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:55:31.592693 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d74b260-1883-4732-96d6-93c8b0e8cc2a admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:55:31.592972 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d74b260-1883-4732-96d6-93c8b0e8cc2a admin admin] PATH_INFO: `/v3/users` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:55:31.593644 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d74b260-1883-4732-96d6-93c8b0e8cc2a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:55:31.595589 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d74b260-1883-4732-96d6-93c8b0e8cc2a admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:55:31.616093 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1322/2619] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Jul 27 10:55:31 2022] POST /identity/v3/users => generated 288 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 10:55:31.619983 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5062bdb-9ab4-4056-a5d9-4da0d64bde54 admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:55:31.630788 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5062bdb-9ab4-4056-a5d9-4da0d64bde54 admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:55:31.635809 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ff92f1-a1b8-4e4d-b397-23b9c2e130ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:55:31.636420 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ff92f1-a1b8-4e4d-b397-23b9c2e130ca admin admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:55:31.636702 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ff92f1-a1b8-4e4d-b397-23b9c2e130ca admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:55:31.636974 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ff92f1-a1b8-4e4d-b397-23b9c2e130ca admin admin] PATH_INFO: `/v3/roles` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:55:31.637655 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ff92f1-a1b8-4e4d-b397-23b9c2e130ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:55:31.639282 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ff92f1-a1b8-4e4d-b397-23b9c2e130ca admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:55:31.651387 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1298/2620] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 10:55:31 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 10:55:31.655304 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d74b260-1883-4732-96d6-93c8b0e8cc2a admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:55:31.665156 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d74b260-1883-4732-96d6-93c8b0e8cc2a admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:55:31.669760 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca79cd15-d35a-481c-a018-ed3e04fa2b53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:55:31.670354 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca79cd15-d35a-481c-a018-ed3e04fa2b53 admin admin] REQUEST_METHOD: `PUT` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:55:31.670664 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca79cd15-d35a-481c-a018-ed3e04fa2b53 admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:55:31.670945 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca79cd15-d35a-481c-a018-ed3e04fa2b53 admin admin] PATH_INFO: `/v3/projects/6106a88006b546daae3bfeaf0a25203d/users/4e3a5824b24f48209aa3987fdcea9a0b/roles/21156560eb6b436bb567a40b4976f3f4` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:55:31.685143 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca79cd15-d35a-481c-a018-ed3e04fa2b53 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6106a88006b546daae3bfeaf0a25203d, user_id=4e3a5824b24f48209aa3987fdcea9a0b, role_id=21156560eb6b436bb567a40b4976f3f4)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:55:31.687807 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca79cd15-d35a-481c-a018-ed3e04fa2b53 admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:55:31.701743 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1323/2621] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 10:55:31 2022] PUT /identity/v3/projects/6106a88006b546daae3bfeaf0a25203d/users/4e3a5824b24f48209aa3987fdcea9a0b/roles/21156560eb6b436bb567a40b4976f3f4 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 10:55:31.705658 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ff92f1-a1b8-4e4d-b397-23b9c2e130ca admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:55:31.716541 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ff92f1-a1b8-4e4d-b397-23b9c2e130ca admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:55:31.720178 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:55:31.775750 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf363cf4-c2a7-439e-b73b-2c69c4433f6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:55:31.776381 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf363cf4-c2a7-439e-b73b-2c69c4433f6d admin admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:55:31.776671 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf363cf4-c2a7-439e-b73b-2c69c4433f6d admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:55:31.776947 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf363cf4-c2a7-439e-b73b-2c69c4433f6d admin admin] PATH_INFO: `/v3/roles` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:55:31.777638 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf363cf4-c2a7-439e-b73b-2c69c4433f6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:55:31.779269 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf363cf4-c2a7-439e-b73b-2c69c4433f6d admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:55:31.788980 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1299/2622] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 10:55:31 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 10:55:31.793164 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca79cd15-d35a-481c-a018-ed3e04fa2b53 admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:55:31.803498 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca79cd15-d35a-481c-a018-ed3e04fa2b53 admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:55:31.807998 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2fe3f0-1ca4-48b9-92c1-53bd41d70100 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:55:31.808651 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2fe3f0-1ca4-48b9-92c1-53bd41d70100 admin admin] REQUEST_METHOD: `PUT` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:55:31.808935 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2fe3f0-1ca4-48b9-92c1-53bd41d70100 admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:55:31.809208 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2fe3f0-1ca4-48b9-92c1-53bd41d70100 admin admin] PATH_INFO: `/v3/projects/6106a88006b546daae3bfeaf0a25203d/users/4e3a5824b24f48209aa3987fdcea9a0b/roles/21156560eb6b436bb567a40b4976f3f4` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:55:31.811178 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2fe3f0-1ca4-48b9-92c1-53bd41d70100 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6106a88006b546daae3bfeaf0a25203d, user_id=4e3a5824b24f48209aa3987fdcea9a0b, role_id=21156560eb6b436bb567a40b4976f3f4)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:55:31.813774 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2fe3f0-1ca4-48b9-92c1-53bd41d70100 admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:55:31.824212 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1324/2623] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 10:55:31 2022] PUT /identity/v3/projects/6106a88006b546daae3bfeaf0a25203d/users/4e3a5824b24f48209aa3987fdcea9a0b/roles/21156560eb6b436bb567a40b4976f3f4 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:55:31.835655 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cade311-55b1-49fd-bbac-188d4d4d47a0 None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:55:31.835942 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cade311-55b1-49fd-bbac-188d4d4d47a0 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:55:31.836220 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cade311-55b1-49fd-bbac-188d4d4d47a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:55:31.863476 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-2cade311-55b1-49fd-bbac-188d4d4d47a0 None None] MFA Rules not processed for user `4e3a5824b24f48209aa3987fdcea9a0b`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:55:31.864663 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-2cade311-55b1-49fd-bbac-188d4d4d47a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:55:31.930675 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1300/2624] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:55:31 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:55:31.942242 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2fe3f0-1ca4-48b9-92c1-53bd41d70100 admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:55:31.943967 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-ec2fe3f0-1ca4-48b9-92c1-53bd41d70100 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:55:32.002482 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2fe3f0-1ca4-48b9-92c1-53bd41d70100 admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:55:32.006082 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b2c534-faee-41b4-9b78-e82793d99b69 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '864d35f268cf45148756109759bb2743', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:55:32.006725 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b2c534-faee-41b4-9b78-e82793d99b69 service cinder] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:55:32.007007 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b2c534-faee-41b4-9b78-e82793d99b69 service cinder] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:55:32.007278 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b2c534-faee-41b4-9b78-e82793d99b69 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:55:32.016685 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b2c534-faee-41b4-9b78-e82793d99b69 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:55:32.018094 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b2c534-faee-41b4-9b78-e82793d99b69 service cinder] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:55:32.027692 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1325/2625] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:55:31 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 10:55:32.103472 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cade311-55b1-49fd-bbac-188d4d4d47a0 None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:55:32.113179 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cade311-55b1-49fd-bbac-188d4d4d47a0 None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:55:32.117215 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22de983c-81a4-4e62-adbc-1d4ed026b2ec service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '864d35f268cf45148756109759bb2743', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:55:32.117834 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22de983c-81a4-4e62-adbc-1d4ed026b2ec service cinder] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:55:32.118118 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22de983c-81a4-4e62-adbc-1d4ed026b2ec service cinder] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:55:32.118918 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22de983c-81a4-4e62-adbc-1d4ed026b2ec service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:55:32.178518 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22de983c-81a4-4e62-adbc-1d4ed026b2ec service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:55:32.179998 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22de983c-81a4-4e62-adbc-1d4ed026b2ec service cinder] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:55:32.192666 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1301/2626] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:55:32 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 10:55:34.973394 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b2c534-faee-41b4-9b78-e82793d99b69 service cinder] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:55:35.008800 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:55:35.043764 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b2c534-faee-41b4-9b78-e82793d99b69 service cinder] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:55:35.048370 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91017ea-ad24-4ace-9c0f-4f881c8ae492 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:55:35.049009 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91017ea-ad24-4ace-9c0f-4f881c8ae492 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:55:35.049332 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91017ea-ad24-4ace-9c0f-4f881c8ae492 admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:55:35.049588 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91017ea-ad24-4ace-9c0f-4f881c8ae492 admin admin] PATH_INFO: `/v3/users/1b8cc040da10482e936e018ef2a5e194` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:55:35.050785 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91017ea-ad24-4ace-9c0f-4f881c8ae492 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1b8cc040da10482e936e018ef2a5e194)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:55:35.052752 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91017ea-ad24-4ace-9c0f-4f881c8ae492 admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:55:35.097508 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-a91017ea-ad24-4ace-9c0f-4f881c8ae492 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1b8cc040da10482e936e018ef2a5e194'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:55:35.104258 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-a91017ea-ad24-4ace-9c0f-4f881c8ae492 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1b8cc040da10482e936e018ef2a5e194'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:55:35.111138 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-a91017ea-ad24-4ace-9c0f-4f881c8ae492 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1b8cc040da10482e936e018ef2a5e194'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:55:35.111754 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-a91017ea-ad24-4ace-9c0f-4f881c8ae492 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1b8cc040da10482e936e018ef2a5e194'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:55:35.112352 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-a91017ea-ad24-4ace-9c0f-4f881c8ae492 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1b8cc040da10482e936e018ef2a5e194'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:55:35.130486 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1326/2627] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Jul 27 10:55:34 2022] DELETE /identity/v3/users/1b8cc040da10482e936e018ef2a5e194 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:55:35.135342 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138b63ac-21ba-4734-b8ca-e929dfb811a3 None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:55:35.135804 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138b63ac-21ba-4734-b8ca-e929dfb811a3 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:55:35.135901 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138b63ac-21ba-4734-b8ca-e929dfb811a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:55:35.154817 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-138b63ac-21ba-4734-b8ca-e929dfb811a3 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:55:35.156019 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-138b63ac-21ba-4734-b8ca-e929dfb811a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:55:35.176375 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:55:35.215802 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1302/2628] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:55:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 10:55:35.225396 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91017ea-ad24-4ace-9c0f-4f881c8ae492 admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:55:35.226855 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-a91017ea-ad24-4ace-9c0f-4f881c8ae492 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:55:35.285463 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91017ea-ad24-4ace-9c0f-4f881c8ae492 admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:55:35.289242 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba9c32b-3b97-43b9-8158-50b213105b6a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7922d0a1a7c4e808d0b2a80dca0da40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:55:35.289867 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba9c32b-3b97-43b9-8158-50b213105b6a service neutron] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:55:35.290174 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba9c32b-3b97-43b9-8158-50b213105b6a service neutron] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:55:35.290437 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba9c32b-3b97-43b9-8158-50b213105b6a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:55:35.301024 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba9c32b-3b97-43b9-8158-50b213105b6a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:55:35.302468 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba9c32b-3b97-43b9-8158-50b213105b6a service neutron] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:55:35.313574 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1327/2629] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:55:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 10:55:35.595523 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138b63ac-21ba-4734-b8ca-e929dfb811a3 None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:55:35.596957 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-138b63ac-21ba-4734-b8ca-e929dfb811a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:55:35.608609 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138b63ac-21ba-4734-b8ca-e929dfb811a3 None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:55:35.613354 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4e47a8-719e-45a8-b8d5-af8e4fbfa332 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:55:35.614013 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4e47a8-719e-45a8-b8d5-af8e4fbfa332 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:55:35.614294 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4e47a8-719e-45a8-b8d5-af8e4fbfa332 admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:55:35.614567 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4e47a8-719e-45a8-b8d5-af8e4fbfa332 admin admin] PATH_INFO: `/v3/projects/367fba0042ce43c79cdaaac707c07f2f` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:55:35.615766 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4e47a8-719e-45a8-b8d5-af8e4fbfa332 admin admin] RBAC: Authorizing `identity:delete_project(project_id=367fba0042ce43c79cdaaac707c07f2f)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:55:35.617674 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4e47a8-719e-45a8-b8d5-af8e4fbfa332 admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:55:35.669337 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-5a4e47a8-719e-45a8-b8d5-af8e4fbfa332 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '367fba0042ce43c79cdaaac707c07f2f', 'request_id': 'req-5a4e47a8-719e-45a8-b8d5-af8e4fbfa332'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:55:35.675785 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-5a4e47a8-719e-45a8-b8d5-af8e4fbfa332 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '367fba0042ce43c79cdaaac707c07f2f', 'request_id': 'req-5a4e47a8-719e-45a8-b8d5-af8e4fbfa332'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:55:35.684145 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-5a4e47a8-719e-45a8-b8d5-af8e4fbfa332 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '367fba0042ce43c79cdaaac707c07f2f', 'request_id': 'req-5a4e47a8-719e-45a8-b8d5-af8e4fbfa332'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:55:35.694742 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-5a4e47a8-719e-45a8-b8d5-af8e4fbfa332 admin admin] The token cache is being invalidate because project 367fba0042ce43c79cdaaac707c07f2f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86828) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 10:55:35.695038 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-5a4e47a8-719e-45a8-b8d5-af8e4fbfa332 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:55:35.695662 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-5a4e47a8-719e-45a8-b8d5-af8e4fbfa332 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:55:35.697814 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1303/2630] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Jul 27 10:55:35 2022] DELETE /identity/v3/projects/367fba0042ce43c79cdaaac707c07f2f => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:55:35.701784 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba9c32b-3b97-43b9-8158-50b213105b6a service neutron] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:55:35.712448 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba9c32b-3b97-43b9-8158-50b213105b6a service neutron] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:55:35.760534 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:55:35.771872 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cc9b2b-f9e1-467a-b1e0-29c8a9036703 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:55:35.772509 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cc9b2b-f9e1-467a-b1e0-29c8a9036703 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:55:35.772788 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cc9b2b-f9e1-467a-b1e0-29c8a9036703 admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:55:35.773060 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cc9b2b-f9e1-467a-b1e0-29c8a9036703 admin admin] PATH_INFO: `/v3/users/4e3a5824b24f48209aa3987fdcea9a0b` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:55:35.774204 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00cc9b2b-f9e1-467a-b1e0-29c8a9036703 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4e3a5824b24f48209aa3987fdcea9a0b)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:55:35.776167 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00cc9b2b-f9e1-467a-b1e0-29c8a9036703 admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:55:35.821167 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-00cc9b2b-f9e1-467a-b1e0-29c8a9036703 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4e3a5824b24f48209aa3987fdcea9a0b'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:55:35.828636 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-00cc9b2b-f9e1-467a-b1e0-29c8a9036703 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4e3a5824b24f48209aa3987fdcea9a0b'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:55:35.835933 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-00cc9b2b-f9e1-467a-b1e0-29c8a9036703 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4e3a5824b24f48209aa3987fdcea9a0b'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:55:35.836640 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-00cc9b2b-f9e1-467a-b1e0-29c8a9036703 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4e3a5824b24f48209aa3987fdcea9a0b'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:55:35.837362 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-00cc9b2b-f9e1-467a-b1e0-29c8a9036703 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4e3a5824b24f48209aa3987fdcea9a0b'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:55:35.854461 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1328/2631] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Jul 27 10:55:35 2022] DELETE /identity/v3/users/4e3a5824b24f48209aa3987fdcea9a0b => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:55:36.141871 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4e47a8-719e-45a8-b8d5-af8e4fbfa332 admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:55:36.144509 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-5a4e47a8-719e-45a8-b8d5-af8e4fbfa332 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:55:36.162952 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4e47a8-719e-45a8-b8d5-af8e4fbfa332 admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:55:36.254619 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34170951-3e2b-4a16-9d47-a460b9092049 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:55:36.255283 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34170951-3e2b-4a16-9d47-a460b9092049 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:55:36.255570 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34170951-3e2b-4a16-9d47-a460b9092049 admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:55:36.255849 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34170951-3e2b-4a16-9d47-a460b9092049 admin admin] PATH_INFO: `/v3/projects/6106a88006b546daae3bfeaf0a25203d` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:55:36.257018 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34170951-3e2b-4a16-9d47-a460b9092049 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6106a88006b546daae3bfeaf0a25203d)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:55:36.259063 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34170951-3e2b-4a16-9d47-a460b9092049 admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:55:36.306713 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-34170951-3e2b-4a16-9d47-a460b9092049 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6106a88006b546daae3bfeaf0a25203d', 'request_id': 'req-34170951-3e2b-4a16-9d47-a460b9092049'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:55:36.308574 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:55:36.313717 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-34170951-3e2b-4a16-9d47-a460b9092049 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6106a88006b546daae3bfeaf0a25203d', 'request_id': 'req-34170951-3e2b-4a16-9d47-a460b9092049'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:55:36.321706 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-34170951-3e2b-4a16-9d47-a460b9092049 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6106a88006b546daae3bfeaf0a25203d', 'request_id': 'req-34170951-3e2b-4a16-9d47-a460b9092049'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:55:36.332349 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-34170951-3e2b-4a16-9d47-a460b9092049 admin admin] The token cache is being invalidate because project 6106a88006b546daae3bfeaf0a25203d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86828) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 10:55:36.332646 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-34170951-3e2b-4a16-9d47-a460b9092049 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:55:36.333267 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-34170951-3e2b-4a16-9d47-a460b9092049 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:55:36.334809 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1304/2632] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Jul 27 10:55:36 2022] DELETE /identity/v3/projects/6106a88006b546daae3bfeaf0a25203d => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:55:36.339907 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22445e3c-f7c0-44b5-8711-29923f5d629c None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:55:36.340375 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22445e3c-f7c0-44b5-8711-29923f5d629c None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:55:36.340481 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22445e3c-f7c0-44b5-8711-29923f5d629c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:55:36.361024 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-22445e3c-f7c0-44b5-8711-29923f5d629c None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:55:36.362232 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-22445e3c-f7c0-44b5-8711-29923f5d629c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:55:36.420216 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1329/2633] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:55:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:55:36.427788 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93442802-69e1-4a21-a9bb-b0b53d17f23c None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:55:36.428077 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93442802-69e1-4a21-a9bb-b0b53d17f23c None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:55:36.428355 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93442802-69e1-4a21-a9bb-b0b53d17f23c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:55:36.448605 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-93442802-69e1-4a21-a9bb-b0b53d17f23c None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:55:36.449816 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-93442802-69e1-4a21-a9bb-b0b53d17f23c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:55:36.458660 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1305/2634] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:55:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:55:36.463681 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22445e3c-f7c0-44b5-8711-29923f5d629c None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:55:36.473638 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22445e3c-f7c0-44b5-8711-29923f5d629c None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:55:36.478838 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534345a0-fc79-4724-aca8-bde0b12b9368 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:55:36.479440 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534345a0-fc79-4724-aca8-bde0b12b9368 admin admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:55:36.479723 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534345a0-fc79-4724-aca8-bde0b12b9368 admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:55:36.479997 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534345a0-fc79-4724-aca8-bde0b12b9368 admin admin] PATH_INFO: `/v3/domains` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:55:36.480678 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534345a0-fc79-4724-aca8-bde0b12b9368 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86827) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:55:36.480979 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534345a0-fc79-4724-aca8-bde0b12b9368 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:55:36.482562 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534345a0-fc79-4724-aca8-bde0b12b9368 admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:55:36.493994 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1330/2635] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Jul 27 10:55:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:55:36.498528 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93442802-69e1-4a21-a9bb-b0b53d17f23c None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:55:36.508749 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93442802-69e1-4a21-a9bb-b0b53d17f23c None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:55:36.513491 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430df847-4a90-484d-bda3-7d02c69a7e23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:55:36.514105 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430df847-4a90-484d-bda3-7d02c69a7e23 admin admin] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:55:36.514390 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430df847-4a90-484d-bda3-7d02c69a7e23 admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:55:36.514696 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430df847-4a90-484d-bda3-7d02c69a7e23 admin admin] PATH_INFO: `/v3/projects` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:55:36.515371 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430df847-4a90-484d-bda3-7d02c69a7e23 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:55:36.517315 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430df847-4a90-484d-bda3-7d02c69a7e23 admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:55:36.544690 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1306/2636] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Jul 27 10:55:36 2022] POST /identity/v3/projects => generated 387 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 10:55:36.548894 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534345a0-fc79-4724-aca8-bde0b12b9368 admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:55:36.559055 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534345a0-fc79-4724-aca8-bde0b12b9368 admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:55:36.608666 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:55:36.615207 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c41809-7176-49e8-a1b4-957958408266 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:55:36.615949 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c41809-7176-49e8-a1b4-957958408266 admin admin] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:55:36.616276 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c41809-7176-49e8-a1b4-957958408266 admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:55:36.616864 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c41809-7176-49e8-a1b4-957958408266 admin admin] PATH_INFO: `/v3/users` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:55:36.617612 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c41809-7176-49e8-a1b4-957958408266 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:55:36.620252 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c41809-7176-49e8-a1b4-957958408266 admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:55:36.640986 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1331/2637] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Jul 27 10:55:36 2022] POST /identity/v3/users => generated 294 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 10:55:36.645293 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430df847-4a90-484d-bda3-7d02c69a7e23 admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:55:36.656016 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430df847-4a90-484d-bda3-7d02c69a7e23 admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:55:36.660529 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6a740a-fe9a-405c-9d3e-8b3c6e316c24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:55:36.661143 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6a740a-fe9a-405c-9d3e-8b3c6e316c24 admin admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:55:36.661424 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6a740a-fe9a-405c-9d3e-8b3c6e316c24 admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:55:36.661695 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6a740a-fe9a-405c-9d3e-8b3c6e316c24 admin admin] PATH_INFO: `/v3/roles` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:55:36.662378 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6a740a-fe9a-405c-9d3e-8b3c6e316c24 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:55:36.664090 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6a740a-fe9a-405c-9d3e-8b3c6e316c24 admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:55:36.674917 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1307/2638] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 10:55:36 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 10:55:36.678862 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c41809-7176-49e8-a1b4-957958408266 admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:55:36.689208 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c41809-7176-49e8-a1b4-957958408266 admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:55:36.695276 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ef75d7-10a1-426a-ab1b-8514052fef2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:55:36.695919 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ef75d7-10a1-426a-ab1b-8514052fef2a admin admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:55:36.696188 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ef75d7-10a1-426a-ab1b-8514052fef2a admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:55:36.696497 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ef75d7-10a1-426a-ab1b-8514052fef2a admin admin] PATH_INFO: `/v3/roles` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:55:36.697198 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ef75d7-10a1-426a-ab1b-8514052fef2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:55:36.698857 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ef75d7-10a1-426a-ab1b-8514052fef2a admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:55:36.713042 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1332/2639] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 10:55:36 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 10:55:36.717257 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6a740a-fe9a-405c-9d3e-8b3c6e316c24 admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:55:36.728725 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6a740a-fe9a-405c-9d3e-8b3c6e316c24 admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:55:36.734009 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8d469d-a3da-4057-b677-652c1331a4e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:55:36.734751 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8d469d-a3da-4057-b677-652c1331a4e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:55:36.735039 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8d469d-a3da-4057-b677-652c1331a4e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:55:36.735325 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8d469d-a3da-4057-b677-652c1331a4e8 admin admin] PATH_INFO: `/v3/projects/36b1a3681133496dbe21982630c59b60/users/0095926e2c7d4029904d5496f4504922/roles/4c9fc0b3baed441da2126be41cc6b04a` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:55:36.752184 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8d469d-a3da-4057-b677-652c1331a4e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=36b1a3681133496dbe21982630c59b60, user_id=0095926e2c7d4029904d5496f4504922, role_id=4c9fc0b3baed441da2126be41cc6b04a)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:55:36.754923 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8d469d-a3da-4057-b677-652c1331a4e8 admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:55:36.767491 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1308/2640] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 10:55:36 2022] PUT /identity/v3/projects/36b1a3681133496dbe21982630c59b60/users/0095926e2c7d4029904d5496f4504922/roles/4c9fc0b3baed441da2126be41cc6b04a => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:55:36.778375 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74cf210-e386-4e3a-a94c-a64af5be2df0 None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:55:36.778699 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74cf210-e386-4e3a-a94c-a64af5be2df0 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:55:36.778979 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74cf210-e386-4e3a-a94c-a64af5be2df0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:55:36.806199 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-c74cf210-e386-4e3a-a94c-a64af5be2df0 None None] MFA Rules not processed for user `0095926e2c7d4029904d5496f4504922`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:55:36.807421 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-c74cf210-e386-4e3a-a94c-a64af5be2df0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:55:36.871439 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1333/2641] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:55:36 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:55:36.876481 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8d469d-a3da-4057-b677-652c1331a4e8 admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:55:37.067982 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8d469d-a3da-4057-b677-652c1331a4e8 admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:55:37.073390 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f311d66-234b-4f17-a29a-853ca447624c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:55:37.074021 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f311d66-234b-4f17-a29a-853ca447624c admin admin] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:55:37.074308 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f311d66-234b-4f17-a29a-853ca447624c admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:55:37.074587 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f311d66-234b-4f17-a29a-853ca447624c admin admin] PATH_INFO: `/v3/projects` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:55:37.075307 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f311d66-234b-4f17-a29a-853ca447624c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:55:37.077179 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f311d66-234b-4f17-a29a-853ca447624c admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:55:37.104898 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1309/2642] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Jul 27 10:55:36 2022] POST /identity/v3/projects => generated 385 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 10:55:37.108688 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74cf210-e386-4e3a-a94c-a64af5be2df0 None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:55:37.119829 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74cf210-e386-4e3a-a94c-a64af5be2df0 None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:55:37.179633 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00488c6d-ab1b-431a-a416-a0c6e376524c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:55:37.180272 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00488c6d-ab1b-431a-a416-a0c6e376524c admin admin] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:55:37.180557 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00488c6d-ab1b-431a-a416-a0c6e376524c admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:55:37.181361 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00488c6d-ab1b-431a-a416-a0c6e376524c admin admin] PATH_INFO: `/v3/users` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:55:37.182051 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00488c6d-ab1b-431a-a416-a0c6e376524c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:55:37.184011 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00488c6d-ab1b-431a-a416-a0c6e376524c admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:55:37.208505 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1334/2643] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Jul 27 10:55:37 2022] POST /identity/v3/users => generated 299 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 10:55:37.209762 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f311d66-234b-4f17-a29a-853ca447624c admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:55:37.220329 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f311d66-234b-4f17-a29a-853ca447624c admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:55:37.225391 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ba2ddd-6204-4ae0-b1fa-aba1416533f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:55:37.226013 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ba2ddd-6204-4ae0-b1fa-aba1416533f3 admin admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:55:37.226295 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ba2ddd-6204-4ae0-b1fa-aba1416533f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:55:37.226723 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ba2ddd-6204-4ae0-b1fa-aba1416533f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:55:37.227341 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ba2ddd-6204-4ae0-b1fa-aba1416533f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:55:37.229048 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ba2ddd-6204-4ae0-b1fa-aba1416533f3 admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:55:37.244872 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1310/2644] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 10:55:37 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 10:55:37.245785 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00488c6d-ab1b-431a-a416-a0c6e376524c admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:55:37.257797 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00488c6d-ab1b-431a-a416-a0c6e376524c admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:55:37.265199 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83df80fd-b125-48f9-8f50-0c6e92b78c12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:55:37.265888 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83df80fd-b125-48f9-8f50-0c6e92b78c12 admin admin] REQUEST_METHOD: `PUT` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:55:37.266174 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83df80fd-b125-48f9-8f50-0c6e92b78c12 admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:55:37.266453 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83df80fd-b125-48f9-8f50-0c6e92b78c12 admin admin] PATH_INFO: `/v3/projects/3642ccdb4cc84ff29a4afbe48d52afa5/users/8d42f544ad8642b89f4afdcaa7837a8b/roles/21156560eb6b436bb567a40b4976f3f4` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:55:37.283372 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83df80fd-b125-48f9-8f50-0c6e92b78c12 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3642ccdb4cc84ff29a4afbe48d52afa5, user_id=8d42f544ad8642b89f4afdcaa7837a8b, role_id=21156560eb6b436bb567a40b4976f3f4)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:55:37.286294 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83df80fd-b125-48f9-8f50-0c6e92b78c12 admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:55:37.302321 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1335/2645] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 10:55:37 2022] PUT /identity/v3/projects/3642ccdb4cc84ff29a4afbe48d52afa5/users/8d42f544ad8642b89f4afdcaa7837a8b/roles/21156560eb6b436bb567a40b4976f3f4 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:55:37.307840 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ba2ddd-6204-4ae0-b1fa-aba1416533f3 admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:55:37.317794 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ba2ddd-6204-4ae0-b1fa-aba1416533f3 admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:55:37.381638 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:55:37.397471 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887eb375-36c6-4aa5-9120-8c92d39e10fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:55:37.398241 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887eb375-36c6-4aa5-9120-8c92d39e10fa admin admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:55:37.399190 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887eb375-36c6-4aa5-9120-8c92d39e10fa admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:55:37.399734 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887eb375-36c6-4aa5-9120-8c92d39e10fa admin admin] PATH_INFO: `/v3/roles` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:55:37.400789 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887eb375-36c6-4aa5-9120-8c92d39e10fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:55:37.402669 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887eb375-36c6-4aa5-9120-8c92d39e10fa admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:55:37.417988 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1311/2646] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 10:55:37 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 10:55:37.422331 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83df80fd-b125-48f9-8f50-0c6e92b78c12 admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:55:37.433634 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83df80fd-b125-48f9-8f50-0c6e92b78c12 admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:55:37.438547 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5739b7d-61c2-4437-9a6f-2951d9e2a010 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:55:37.439356 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5739b7d-61c2-4437-9a6f-2951d9e2a010 admin admin] REQUEST_METHOD: `PUT` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:55:37.439752 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5739b7d-61c2-4437-9a6f-2951d9e2a010 admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:55:37.440124 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5739b7d-61c2-4437-9a6f-2951d9e2a010 admin admin] PATH_INFO: `/v3/projects/3642ccdb4cc84ff29a4afbe48d52afa5/users/8d42f544ad8642b89f4afdcaa7837a8b/roles/21156560eb6b436bb567a40b4976f3f4` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:55:37.442260 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5739b7d-61c2-4437-9a6f-2951d9e2a010 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3642ccdb4cc84ff29a4afbe48d52afa5, user_id=8d42f544ad8642b89f4afdcaa7837a8b, role_id=21156560eb6b436bb567a40b4976f3f4)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:55:37.445149 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5739b7d-61c2-4437-9a6f-2951d9e2a010 admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:55:37.459614 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1336/2647] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 10:55:37 2022] PUT /identity/v3/projects/3642ccdb4cc84ff29a4afbe48d52afa5/users/8d42f544ad8642b89f4afdcaa7837a8b/roles/21156560eb6b436bb567a40b4976f3f4 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:55:37.471316 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d20027d-d521-4266-9c77-ebdbf9624459 None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:55:37.471938 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d20027d-d521-4266-9c77-ebdbf9624459 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:55:37.472401 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d20027d-d521-4266-9c77-ebdbf9624459 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:55:37.523947 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-7d20027d-d521-4266-9c77-ebdbf9624459 None None] MFA Rules not processed for user `8d42f544ad8642b89f4afdcaa7837a8b`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:55:37.525300 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-7d20027d-d521-4266-9c77-ebdbf9624459 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:55:37.596629 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1312/2648] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:55:37 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:55:37.608696 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5739b7d-61c2-4437-9a6f-2951d9e2a010 admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:55:37.610445 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-c5739b7d-61c2-4437-9a6f-2951d9e2a010 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:55:37.632374 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:55:37.670860 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5739b7d-61c2-4437-9a6f-2951d9e2a010 admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:55:37.674879 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af4c57e-4596-4317-917b-5357b3dfa12b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '864d35f268cf45148756109759bb2743', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:55:37.675398 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af4c57e-4596-4317-917b-5357b3dfa12b service cinder] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:55:37.675683 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af4c57e-4596-4317-917b-5357b3dfa12b service cinder] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:55:37.675959 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af4c57e-4596-4317-917b-5357b3dfa12b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:55:37.685700 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af4c57e-4596-4317-917b-5357b3dfa12b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:55:37.687203 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af4c57e-4596-4317-917b-5357b3dfa12b service cinder] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:55:37.696822 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1337/2649] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:55:37 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 10:55:37.725706 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d20027d-d521-4266-9c77-ebdbf9624459 None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:55:37.790916 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d20027d-d521-4266-9c77-ebdbf9624459 None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:55:37.795556 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548c484c-9a29-47e8-8d8a-7f7ada657cf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:55:37.796202 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548c484c-9a29-47e8-8d8a-7f7ada657cf0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:55:37.796489 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548c484c-9a29-47e8-8d8a-7f7ada657cf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:55:37.796766 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548c484c-9a29-47e8-8d8a-7f7ada657cf0 admin admin] PATH_INFO: `/v3/users/0095926e2c7d4029904d5496f4504922` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:55:37.797953 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548c484c-9a29-47e8-8d8a-7f7ada657cf0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0095926e2c7d4029904d5496f4504922)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:55:37.800064 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548c484c-9a29-47e8-8d8a-7f7ada657cf0 admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:55:37.846774 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-548c484c-9a29-47e8-8d8a-7f7ada657cf0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0095926e2c7d4029904d5496f4504922'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:55:37.854045 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-548c484c-9a29-47e8-8d8a-7f7ada657cf0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0095926e2c7d4029904d5496f4504922'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:55:37.861662 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-548c484c-9a29-47e8-8d8a-7f7ada657cf0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0095926e2c7d4029904d5496f4504922'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:55:37.862419 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-548c484c-9a29-47e8-8d8a-7f7ada657cf0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0095926e2c7d4029904d5496f4504922'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:55:37.863203 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-548c484c-9a29-47e8-8d8a-7f7ada657cf0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0095926e2c7d4029904d5496f4504922'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:55:37.871957 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:55:37.885139 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1313/2650] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Jul 27 10:55:37 2022] DELETE /identity/v3/users/0095926e2c7d4029904d5496f4504922 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:55:37.889399 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d8766c-28c1-45c8-9b54-d69a866a9073 None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:55:37.889759 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d8766c-28c1-45c8-9b54-d69a866a9073 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:55:37.890104 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d8766c-28c1-45c8-9b54-d69a866a9073 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:55:37.910329 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-b4d8766c-28c1-45c8-9b54-d69a866a9073 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:55:37.911615 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-b4d8766c-28c1-45c8-9b54-d69a866a9073 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:55:37.974135 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1338/2651] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:55:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:55:37.984473 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548c484c-9a29-47e8-8d8a-7f7ada657cf0 admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:55:37.985974 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-548c484c-9a29-47e8-8d8a-7f7ada657cf0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:55:38.046167 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548c484c-9a29-47e8-8d8a-7f7ada657cf0 admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:55:38.049859 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f58b4c-763f-497d-b118-fae247bbb2f6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7922d0a1a7c4e808d0b2a80dca0da40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:55:38.050492 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f58b4c-763f-497d-b118-fae247bbb2f6 service neutron] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:55:38.050847 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f58b4c-763f-497d-b118-fae247bbb2f6 service neutron] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:55:38.051124 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f58b4c-763f-497d-b118-fae247bbb2f6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:55:38.060751 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f58b4c-763f-497d-b118-fae247bbb2f6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:55:38.062181 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f58b4c-763f-497d-b118-fae247bbb2f6 service neutron] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:55:38.072633 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1314/2652] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:55:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 10:55:38.510025 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d8766c-28c1-45c8-9b54-d69a866a9073 None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:55:38.511568 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-b4d8766c-28c1-45c8-9b54-d69a866a9073 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:55:38.527019 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d8766c-28c1-45c8-9b54-d69a866a9073 None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:55:38.532142 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbf382f-98e3-4d12-8684-3bc442f0a651 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:55:38.532829 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbf382f-98e3-4d12-8684-3bc442f0a651 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:55:38.533119 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbf382f-98e3-4d12-8684-3bc442f0a651 admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:55:38.533692 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbf382f-98e3-4d12-8684-3bc442f0a651 admin admin] PATH_INFO: `/v3/projects/36b1a3681133496dbe21982630c59b60` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:55:38.535016 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbf382f-98e3-4d12-8684-3bc442f0a651 admin admin] RBAC: Authorizing `identity:delete_project(project_id=36b1a3681133496dbe21982630c59b60)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:55:38.537873 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbf382f-98e3-4d12-8684-3bc442f0a651 admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:55:38.593027 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-bcbf382f-98e3-4d12-8684-3bc442f0a651 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '36b1a3681133496dbe21982630c59b60', 'request_id': 'req-bcbf382f-98e3-4d12-8684-3bc442f0a651'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:55:38.600095 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-bcbf382f-98e3-4d12-8684-3bc442f0a651 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '36b1a3681133496dbe21982630c59b60', 'request_id': 'req-bcbf382f-98e3-4d12-8684-3bc442f0a651'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:55:38.613787 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-bcbf382f-98e3-4d12-8684-3bc442f0a651 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '36b1a3681133496dbe21982630c59b60', 'request_id': 'req-bcbf382f-98e3-4d12-8684-3bc442f0a651'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:55:38.625723 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-bcbf382f-98e3-4d12-8684-3bc442f0a651 admin admin] The token cache is being invalidate because project 36b1a3681133496dbe21982630c59b60 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86827) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 10:55:38.626009 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-bcbf382f-98e3-4d12-8684-3bc442f0a651 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:55:38.626647 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-bcbf382f-98e3-4d12-8684-3bc442f0a651 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:55:38.628430 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1339/2653] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Jul 27 10:55:38 2022] DELETE /identity/v3/projects/36b1a3681133496dbe21982630c59b60 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:55:38.632365 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f58b4c-763f-497d-b118-fae247bbb2f6 service neutron] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:55:38.641888 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f58b4c-763f-497d-b118-fae247bbb2f6 service neutron] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:55:38.697935 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b89afee-d9fe-44e7-82b9-904d1922b822 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:55:38.698587 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b89afee-d9fe-44e7-82b9-904d1922b822 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:55:38.698898 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b89afee-d9fe-44e7-82b9-904d1922b822 admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:55:38.699734 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b89afee-d9fe-44e7-82b9-904d1922b822 admin admin] PATH_INFO: `/v3/users/8d42f544ad8642b89f4afdcaa7837a8b` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:55:38.700364 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b89afee-d9fe-44e7-82b9-904d1922b822 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8d42f544ad8642b89f4afdcaa7837a8b)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:55:38.702296 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b89afee-d9fe-44e7-82b9-904d1922b822 admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:55:38.754374 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-9b89afee-d9fe-44e7-82b9-904d1922b822 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8d42f544ad8642b89f4afdcaa7837a8b'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:55:38.762076 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-9b89afee-d9fe-44e7-82b9-904d1922b822 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8d42f544ad8642b89f4afdcaa7837a8b'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:55:38.770312 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-9b89afee-d9fe-44e7-82b9-904d1922b822 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8d42f544ad8642b89f4afdcaa7837a8b'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:55:38.771020 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-9b89afee-d9fe-44e7-82b9-904d1922b822 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8d42f544ad8642b89f4afdcaa7837a8b'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:55:38.771644 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-9b89afee-d9fe-44e7-82b9-904d1922b822 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8d42f544ad8642b89f4afdcaa7837a8b'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:55:38.789876 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1315/2654] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Jul 27 10:55:38 2022] DELETE /identity/v3/users/8d42f544ad8642b89f4afdcaa7837a8b => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:55:39.064683 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbf382f-98e3-4d12-8684-3bc442f0a651 admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:55:39.066485 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-bcbf382f-98e3-4d12-8684-3bc442f0a651 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:55:39.083704 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbf382f-98e3-4d12-8684-3bc442f0a651 admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:55:39.139272 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:55:39.157833 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb541fa-e0ae-42c7-a0e3-78eb9c8cf506 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:55:39.158500 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb541fa-e0ae-42c7-a0e3-78eb9c8cf506 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:55:39.158934 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb541fa-e0ae-42c7-a0e3-78eb9c8cf506 admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:55:39.159246 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb541fa-e0ae-42c7-a0e3-78eb9c8cf506 admin admin] PATH_INFO: `/v3/projects/3642ccdb4cc84ff29a4afbe48d52afa5` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:55:39.160485 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb541fa-e0ae-42c7-a0e3-78eb9c8cf506 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3642ccdb4cc84ff29a4afbe48d52afa5)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:55:39.162427 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb541fa-e0ae-42c7-a0e3-78eb9c8cf506 admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:55:39.213339 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-beb541fa-e0ae-42c7-a0e3-78eb9c8cf506 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3642ccdb4cc84ff29a4afbe48d52afa5', 'request_id': 'req-beb541fa-e0ae-42c7-a0e3-78eb9c8cf506'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:55:39.219847 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-beb541fa-e0ae-42c7-a0e3-78eb9c8cf506 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3642ccdb4cc84ff29a4afbe48d52afa5', 'request_id': 'req-beb541fa-e0ae-42c7-a0e3-78eb9c8cf506'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:55:39.229703 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-beb541fa-e0ae-42c7-a0e3-78eb9c8cf506 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3642ccdb4cc84ff29a4afbe48d52afa5', 'request_id': 'req-beb541fa-e0ae-42c7-a0e3-78eb9c8cf506'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:55:39.241362 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-beb541fa-e0ae-42c7-a0e3-78eb9c8cf506 admin admin] The token cache is being invalidate because project 3642ccdb4cc84ff29a4afbe48d52afa5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86827) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 10:55:39.241662 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-beb541fa-e0ae-42c7-a0e3-78eb9c8cf506 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:55:39.242364 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-beb541fa-e0ae-42c7-a0e3-78eb9c8cf506 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:55:39.243922 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1340/2655] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Jul 27 10:55:39 2022] DELETE /identity/v3/projects/3642ccdb4cc84ff29a4afbe48d52afa5 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:55:39.248965 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca04aa1-163f-4373-ae5c-4b8894c03e4a None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:55:39.249326 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca04aa1-163f-4373-ae5c-4b8894c03e4a None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:55:39.249675 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca04aa1-163f-4373-ae5c-4b8894c03e4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:55:39.270920 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-6ca04aa1-163f-4373-ae5c-4b8894c03e4a None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:55:39.272222 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-6ca04aa1-163f-4373-ae5c-4b8894c03e4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:55:39.281519 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:55:39.334814 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1316/2656] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:55:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:55:39.343037 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e7df6a-096d-42ff-b72b-3f7e87eb6ee8 None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:55:39.343326 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e7df6a-096d-42ff-b72b-3f7e87eb6ee8 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:55:39.343605 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e7df6a-096d-42ff-b72b-3f7e87eb6ee8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:55:39.362807 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-f7e7df6a-096d-42ff-b72b-3f7e87eb6ee8 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:55:39.364007 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-f7e7df6a-096d-42ff-b72b-3f7e87eb6ee8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:55:39.371825 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1341/2657] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:55:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:55:39.376337 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca04aa1-163f-4373-ae5c-4b8894c03e4a None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:55:39.390370 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca04aa1-163f-4373-ae5c-4b8894c03e4a None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:55:39.395992 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2457ee83-edf5-459d-8b76-bbb3eb5edd2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:55:39.396618 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2457ee83-edf5-459d-8b76-bbb3eb5edd2a admin admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:55:39.396898 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2457ee83-edf5-459d-8b76-bbb3eb5edd2a admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:55:39.397170 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2457ee83-edf5-459d-8b76-bbb3eb5edd2a admin admin] PATH_INFO: `/v3/domains` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:55:39.397872 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2457ee83-edf5-459d-8b76-bbb3eb5edd2a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86828) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:55:39.398177 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2457ee83-edf5-459d-8b76-bbb3eb5edd2a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:55:39.400349 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2457ee83-edf5-459d-8b76-bbb3eb5edd2a admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:55:39.412507 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1317/2658] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Jul 27 10:55:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 10:55:39.416920 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e7df6a-096d-42ff-b72b-3f7e87eb6ee8 None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:55:39.426323 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e7df6a-096d-42ff-b72b-3f7e87eb6ee8 None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:55:39.431511 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbff426-5bf7-4f36-b1b5-9d4cde210e6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:55:39.432127 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbff426-5bf7-4f36-b1b5-9d4cde210e6b admin admin] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:55:39.432412 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbff426-5bf7-4f36-b1b5-9d4cde210e6b admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:55:39.432691 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbff426-5bf7-4f36-b1b5-9d4cde210e6b admin admin] PATH_INFO: `/v3/projects` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:55:39.433354 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbff426-5bf7-4f36-b1b5-9d4cde210e6b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:55:39.435284 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbff426-5bf7-4f36-b1b5-9d4cde210e6b admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:55:39.464562 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1342/2659] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Jul 27 10:55:39 2022] POST /identity/v3/projects => generated 373 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 10:55:39.468731 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2457ee83-edf5-459d-8b76-bbb3eb5edd2a admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:55:39.479595 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2457ee83-edf5-459d-8b76-bbb3eb5edd2a admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:55:39.539915 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a3705a-0f59-4188-88d5-b5fc2f13bafc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:55:39.540563 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a3705a-0f59-4188-88d5-b5fc2f13bafc admin admin] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:55:39.540845 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a3705a-0f59-4188-88d5-b5fc2f13bafc admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:55:39.541119 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a3705a-0f59-4188-88d5-b5fc2f13bafc admin admin] PATH_INFO: `/v3/users` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:55:39.541782 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a3705a-0f59-4188-88d5-b5fc2f13bafc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:55:39.543724 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a3705a-0f59-4188-88d5-b5fc2f13bafc admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:55:39.571282 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1318/2660] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Jul 27 10:55:39 2022] POST /identity/v3/users => generated 287 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 10:55:39.572562 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbff426-5bf7-4f36-b1b5-9d4cde210e6b admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:55:39.584214 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbff426-5bf7-4f36-b1b5-9d4cde210e6b admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:55:39.592902 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd53307-e145-4467-89fe-b5924f662143 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:55:39.593548 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd53307-e145-4467-89fe-b5924f662143 admin admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:55:39.593818 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd53307-e145-4467-89fe-b5924f662143 admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:55:39.594106 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd53307-e145-4467-89fe-b5924f662143 admin admin] PATH_INFO: `/v3/roles` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:55:39.594834 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd53307-e145-4467-89fe-b5924f662143 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:55:39.596470 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd53307-e145-4467-89fe-b5924f662143 admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:55:39.607357 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1343/2661] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 10:55:39 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 10:55:39.611294 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a3705a-0f59-4188-88d5-b5fc2f13bafc admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:55:39.622122 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a3705a-0f59-4188-88d5-b5fc2f13bafc admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:55:39.627544 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0621a9f4-1fd7-4e7e-9df2-00b2da43bbe5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:55:39.628202 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0621a9f4-1fd7-4e7e-9df2-00b2da43bbe5 admin admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:55:39.628483 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0621a9f4-1fd7-4e7e-9df2-00b2da43bbe5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:55:39.628758 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0621a9f4-1fd7-4e7e-9df2-00b2da43bbe5 admin admin] PATH_INFO: `/v3/roles` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:55:39.629455 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0621a9f4-1fd7-4e7e-9df2-00b2da43bbe5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:55:39.631094 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0621a9f4-1fd7-4e7e-9df2-00b2da43bbe5 admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:55:39.643904 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1319/2662] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 10:55:39 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 10:55:39.648079 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd53307-e145-4467-89fe-b5924f662143 admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:55:39.664209 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd53307-e145-4467-89fe-b5924f662143 admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:55:39.668968 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7585b0d7-c216-4ebe-88b2-fa6c40f9a623 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:55:39.669640 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7585b0d7-c216-4ebe-88b2-fa6c40f9a623 admin admin] REQUEST_METHOD: `PUT` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:55:39.669922 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7585b0d7-c216-4ebe-88b2-fa6c40f9a623 admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:55:39.670200 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7585b0d7-c216-4ebe-88b2-fa6c40f9a623 admin admin] PATH_INFO: `/v3/projects/7b9629060ff841feaf09e03396ee02db/users/968007c31fde4b879da9b3ae9a1335aa/roles/4c9fc0b3baed441da2126be41cc6b04a` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:55:39.685246 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7585b0d7-c216-4ebe-88b2-fa6c40f9a623 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b9629060ff841feaf09e03396ee02db, user_id=968007c31fde4b879da9b3ae9a1335aa, role_id=4c9fc0b3baed441da2126be41cc6b04a)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:55:39.687942 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7585b0d7-c216-4ebe-88b2-fa6c40f9a623 admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:55:39.701355 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1344/2663] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 10:55:39 2022] PUT /identity/v3/projects/7b9629060ff841feaf09e03396ee02db/users/968007c31fde4b879da9b3ae9a1335aa/roles/4c9fc0b3baed441da2126be41cc6b04a => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:55:39.706508 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acce640-5113-487a-915d-500d81bf76e5 None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:55:39.706817 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acce640-5113-487a-915d-500d81bf76e5 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:55:39.707093 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acce640-5113-487a-915d-500d81bf76e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:55:39.726254 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-0acce640-5113-487a-915d-500d81bf76e5 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:55:39.727501 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-0acce640-5113-487a-915d-500d81bf76e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:55:39.788685 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1320/2664] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:55:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:55:39.798268 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7585b0d7-c216-4ebe-88b2-fa6c40f9a623 admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:55:39.800273 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-7585b0d7-c216-4ebe-88b2-fa6c40f9a623 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:55:39.860889 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7585b0d7-c216-4ebe-88b2-fa6c40f9a623 admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:55:39.864545 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac95e195-5c3c-4e67-bb82-b8bdbe685cdd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7922d0a1a7c4e808d0b2a80dca0da40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:55:39.865186 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac95e195-5c3c-4e67-bb82-b8bdbe685cdd service neutron] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:55:39.865471 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac95e195-5c3c-4e67-bb82-b8bdbe685cdd service neutron] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:55:39.865749 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac95e195-5c3c-4e67-bb82-b8bdbe685cdd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:55:39.875687 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac95e195-5c3c-4e67-bb82-b8bdbe685cdd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:55:39.877102 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac95e195-5c3c-4e67-bb82-b8bdbe685cdd service neutron] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:55:39.886927 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1345/2665] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:55:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 10:55:40.522528 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d68330-c6ec-4fb5-8481-e8d63a4a4796 None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:55:40.522867 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d68330-c6ec-4fb5-8481-e8d63a4a4796 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:55:40.523143 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d68330-c6ec-4fb5-8481-e8d63a4a4796 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:55:40.559576 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-48d68330-c6ec-4fb5-8481-e8d63a4a4796 None None] MFA Rules not processed for user `968007c31fde4b879da9b3ae9a1335aa`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:55:40.560812 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-48d68330-c6ec-4fb5-8481-e8d63a4a4796 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:55:40.571909 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:55:40.648571 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1321/2666] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:55:40 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:55:40.653357 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac95e195-5c3c-4e67-bb82-b8bdbe685cdd service neutron] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:55:40.665550 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac95e195-5c3c-4e67-bb82-b8bdbe685cdd service neutron] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:55:40.671422 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2235e322-ef35-49df-ae9d-128b98cdcf83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:55:40.672058 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2235e322-ef35-49df-ae9d-128b98cdcf83 admin admin] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:55:40.672345 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2235e322-ef35-49df-ae9d-128b98cdcf83 admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:55:40.672624 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2235e322-ef35-49df-ae9d-128b98cdcf83 admin admin] PATH_INFO: `/v3/projects` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:55:40.673313 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2235e322-ef35-49df-ae9d-128b98cdcf83 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:55:40.675559 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2235e322-ef35-49df-ae9d-128b98cdcf83 admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:55:40.699755 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:55:40.734046 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1346/2667] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Jul 27 10:55:40 2022] POST /identity/v3/projects => generated 369 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 10:55:40.738169 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d68330-c6ec-4fb5-8481-e8d63a4a4796 None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:55:40.750987 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d68330-c6ec-4fb5-8481-e8d63a4a4796 None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:55:40.813778 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782a319f-331b-47df-a67f-7f1450745c5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:55:40.814413 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782a319f-331b-47df-a67f-7f1450745c5d admin admin] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:55:40.814731 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782a319f-331b-47df-a67f-7f1450745c5d admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:55:40.815062 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782a319f-331b-47df-a67f-7f1450745c5d admin admin] PATH_INFO: `/v3/users` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:55:40.815900 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782a319f-331b-47df-a67f-7f1450745c5d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:55:40.817830 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782a319f-331b-47df-a67f-7f1450745c5d admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:55:40.843956 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1322/2668] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Jul 27 10:55:40 2022] POST /identity/v3/users => generated 291 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 10:55:40.847846 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2235e322-ef35-49df-ae9d-128b98cdcf83 admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:55:40.859122 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2235e322-ef35-49df-ae9d-128b98cdcf83 admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:55:40.864114 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61dfe2f5-f796-4e8d-9437-e3e7a3a25cc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:55:40.864897 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dfe2f5-f796-4e8d-9437-e3e7a3a25cc6 admin admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:55:40.865277 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dfe2f5-f796-4e8d-9437-e3e7a3a25cc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:55:40.866033 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dfe2f5-f796-4e8d-9437-e3e7a3a25cc6 admin admin] PATH_INFO: `/v3/roles` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:55:40.866444 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61dfe2f5-f796-4e8d-9437-e3e7a3a25cc6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:55:40.868215 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61dfe2f5-f796-4e8d-9437-e3e7a3a25cc6 admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:55:40.882557 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1347/2669] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 10:55:40 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 10:55:40.887872 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782a319f-331b-47df-a67f-7f1450745c5d admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:55:40.898284 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782a319f-331b-47df-a67f-7f1450745c5d admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:55:40.903555 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab27f9c0-bd2b-4d06-b1e1-8e9e468cc33f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:55:40.904560 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab27f9c0-bd2b-4d06-b1e1-8e9e468cc33f admin admin] REQUEST_METHOD: `PUT` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:55:40.904560 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab27f9c0-bd2b-4d06-b1e1-8e9e468cc33f admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:55:40.904883 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab27f9c0-bd2b-4d06-b1e1-8e9e468cc33f admin admin] PATH_INFO: `/v3/projects/3768dc37162c436f8a446ddb54e121c2/users/61ef1abc462446d182358c3418d02b7b/roles/21156560eb6b436bb567a40b4976f3f4` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:55:40.923177 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab27f9c0-bd2b-4d06-b1e1-8e9e468cc33f admin admin] RBAC: Authorizing `identity:create_grant(project_id=3768dc37162c436f8a446ddb54e121c2, user_id=61ef1abc462446d182358c3418d02b7b, role_id=21156560eb6b436bb567a40b4976f3f4)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:55:40.926136 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab27f9c0-bd2b-4d06-b1e1-8e9e468cc33f admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:55:40.948227 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1323/2670] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 10:55:40 2022] PUT /identity/v3/projects/3768dc37162c436f8a446ddb54e121c2/users/61ef1abc462446d182358c3418d02b7b/roles/21156560eb6b436bb567a40b4976f3f4 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 10:55:40.952279 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61dfe2f5-f796-4e8d-9437-e3e7a3a25cc6 admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:55:40.963574 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61dfe2f5-f796-4e8d-9437-e3e7a3a25cc6 admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:55:41.029264 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031da50d-6bfa-47fd-8b7f-909523190465 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:55:41.030003 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031da50d-6bfa-47fd-8b7f-909523190465 admin admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:55:41.030389 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031da50d-6bfa-47fd-8b7f-909523190465 admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:55:41.030807 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031da50d-6bfa-47fd-8b7f-909523190465 admin admin] PATH_INFO: `/v3/roles` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:55:41.031793 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031da50d-6bfa-47fd-8b7f-909523190465 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:55:41.033471 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031da50d-6bfa-47fd-8b7f-909523190465 admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:55:41.056671 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1348/2671] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 10:55:40 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 10:55:41.070676 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab27f9c0-bd2b-4d06-b1e1-8e9e468cc33f admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:55:41.081744 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab27f9c0-bd2b-4d06-b1e1-8e9e468cc33f admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:55:41.086425 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5be4db5-a57a-4949-b61e-4e41d62f0b4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:55:41.087613 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5be4db5-a57a-4949-b61e-4e41d62f0b4a admin admin] REQUEST_METHOD: `PUT` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:55:41.087894 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5be4db5-a57a-4949-b61e-4e41d62f0b4a admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:55:41.088168 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5be4db5-a57a-4949-b61e-4e41d62f0b4a admin admin] PATH_INFO: `/v3/projects/3768dc37162c436f8a446ddb54e121c2/users/61ef1abc462446d182358c3418d02b7b/roles/21156560eb6b436bb567a40b4976f3f4` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:55:41.090103 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5be4db5-a57a-4949-b61e-4e41d62f0b4a admin admin] RBAC: Authorizing `identity:create_grant(project_id=3768dc37162c436f8a446ddb54e121c2, user_id=61ef1abc462446d182358c3418d02b7b, role_id=21156560eb6b436bb567a40b4976f3f4)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:55:41.092792 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5be4db5-a57a-4949-b61e-4e41d62f0b4a admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:55:41.104754 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1324/2672] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 10:55:41 2022] PUT /identity/v3/projects/3768dc37162c436f8a446ddb54e121c2/users/61ef1abc462446d182358c3418d02b7b/roles/21156560eb6b436bb567a40b4976f3f4 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:55:42.004259 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9feff620-c591-4fb9-8630-a4946d2fdc7d None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:55:42.004800 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9feff620-c591-4fb9-8630-a4946d2fdc7d None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:55:42.004800 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9feff620-c591-4fb9-8630-a4946d2fdc7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:55:42.033652 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-9feff620-c591-4fb9-8630-a4946d2fdc7d None None] MFA Rules not processed for user `61ef1abc462446d182358c3418d02b7b`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:55:42.034895 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-9feff620-c591-4fb9-8630-a4946d2fdc7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:55:42.133601 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1349/2673] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:55:42 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:55:42.145083 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5be4db5-a57a-4949-b61e-4e41d62f0b4a admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:55:42.146803 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-e5be4db5-a57a-4949-b61e-4e41d62f0b4a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:55:42.207096 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5be4db5-a57a-4949-b61e-4e41d62f0b4a admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:55:42.211161 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df4b621-2b3e-406b-bff8-38e03898ab9e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '864d35f268cf45148756109759bb2743', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:55:42.212212 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df4b621-2b3e-406b-bff8-38e03898ab9e service cinder] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:55:42.213208 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df4b621-2b3e-406b-bff8-38e03898ab9e service cinder] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:55:42.213755 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df4b621-2b3e-406b-bff8-38e03898ab9e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:55:42.234376 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:55:42.283076 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df4b621-2b3e-406b-bff8-38e03898ab9e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:55:42.284515 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df4b621-2b3e-406b-bff8-38e03898ab9e service cinder] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:55:42.293904 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1325/2674] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:55:42 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 10:55:43.657527 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9feff620-c591-4fb9-8630-a4946d2fdc7d None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:55:43.658984 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-9feff620-c591-4fb9-8630-a4946d2fdc7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:55:43.671345 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9feff620-c591-4fb9-8630-a4946d2fdc7d None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:55:43.674912 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b11b902-3beb-4ec0-9aa1-34579b2139b4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '864d35f268cf45148756109759bb2743', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:55:43.675515 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b11b902-3beb-4ec0-9aa1-34579b2139b4 service cinder] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:55:43.675816 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b11b902-3beb-4ec0-9aa1-34579b2139b4 service cinder] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:55:43.676074 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b11b902-3beb-4ec0-9aa1-34579b2139b4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:55:43.689367 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b11b902-3beb-4ec0-9aa1-34579b2139b4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:55:43.690822 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b11b902-3beb-4ec0-9aa1-34579b2139b4 service cinder] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:55:43.700268 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1350/2675] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:55:43 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 10:55:50.674976 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df4b621-2b3e-406b-bff8-38e03898ab9e service cinder] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:55:50.676370 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-8df4b621-2b3e-406b-bff8-38e03898ab9e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:55:50.735864 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b11b902-3beb-4ec0-9aa1-34579b2139b4 service cinder] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:55:50.737316 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-2b11b902-3beb-4ec0-9aa1-34579b2139b4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:55:50.757768 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df4b621-2b3e-406b-bff8-38e03898ab9e service cinder] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:55:50.763852 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c16b686-5f53-4f03-8b4e-0da3edef940c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16cc6179e9a04b6398fbacb40ad1c555', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:55:50.764483 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c16b686-5f53-4f03-8b4e-0da3edef940c service placement] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:55:50.764768 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c16b686-5f53-4f03-8b4e-0da3edef940c service placement] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:55:50.765048 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c16b686-5f53-4f03-8b4e-0da3edef940c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:55:50.766878 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-8c16b686-5f53-4f03-8b4e-0da3edef940c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:55:50.790324 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c16b686-5f53-4f03-8b4e-0da3edef940c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:55:50.792000 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:55:50.792518 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c16b686-5f53-4f03-8b4e-0da3edef940c service placement] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:55:50.803246 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1326/2676] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:55:50 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 10:55:50.809086 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b11b902-3beb-4ec0-9aa1-34579b2139b4 service cinder] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:55:50.813689 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff99f57a-6a06-42f3-9fc5-9c03fb96e376 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:55:50.814325 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff99f57a-6a06-42f3-9fc5-9c03fb96e376 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:55:50.814640 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff99f57a-6a06-42f3-9fc5-9c03fb96e376 admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:55:50.815013 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff99f57a-6a06-42f3-9fc5-9c03fb96e376 admin admin] PATH_INFO: `/v3/users/0621626f2863400c903de6a88da84f1e` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:55:50.816091 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff99f57a-6a06-42f3-9fc5-9c03fb96e376 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0621626f2863400c903de6a88da84f1e)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:55:50.818005 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff99f57a-6a06-42f3-9fc5-9c03fb96e376 admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:55:50.865953 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-ff99f57a-6a06-42f3-9fc5-9c03fb96e376 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0621626f2863400c903de6a88da84f1e'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:55:50.873219 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-ff99f57a-6a06-42f3-9fc5-9c03fb96e376 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0621626f2863400c903de6a88da84f1e'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:55:50.880611 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-ff99f57a-6a06-42f3-9fc5-9c03fb96e376 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0621626f2863400c903de6a88da84f1e'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:55:50.881222 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-ff99f57a-6a06-42f3-9fc5-9c03fb96e376 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0621626f2863400c903de6a88da84f1e'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:55:50.881820 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-ff99f57a-6a06-42f3-9fc5-9c03fb96e376 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0621626f2863400c903de6a88da84f1e'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:55:50.899104 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1351/2677] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Jul 27 10:55:50 2022] DELETE /identity/v3/users/0621626f2863400c903de6a88da84f1e => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 10:55:51.070989 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c16b686-5f53-4f03-8b4e-0da3edef940c service placement] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:55:51.125596 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c16b686-5f53-4f03-8b4e-0da3edef940c service placement] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:55:51.176190 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=100 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:55:51.182557 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d10dc9c-76ee-4129-a7ed-0be292277391 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:55:51.183264 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d10dc9c-76ee-4129-a7ed-0be292277391 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:55:51.183547 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d10dc9c-76ee-4129-a7ed-0be292277391 admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:55:51.183845 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d10dc9c-76ee-4129-a7ed-0be292277391 admin admin] PATH_INFO: `/v3/projects/6b50dc16d14d4b869f1a6555db6c8373` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:55:51.185077 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d10dc9c-76ee-4129-a7ed-0be292277391 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6b50dc16d14d4b869f1a6555db6c8373)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:55:51.187028 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d10dc9c-76ee-4129-a7ed-0be292277391 admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:55:51.232203 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-3d10dc9c-76ee-4129-a7ed-0be292277391 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6b50dc16d14d4b869f1a6555db6c8373', 'request_id': 'req-3d10dc9c-76ee-4129-a7ed-0be292277391'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:55:51.238033 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-3d10dc9c-76ee-4129-a7ed-0be292277391 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6b50dc16d14d4b869f1a6555db6c8373', 'request_id': 'req-3d10dc9c-76ee-4129-a7ed-0be292277391'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:55:51.245581 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-3d10dc9c-76ee-4129-a7ed-0be292277391 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6b50dc16d14d4b869f1a6555db6c8373', 'request_id': 'req-3d10dc9c-76ee-4129-a7ed-0be292277391'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:55:51.268427 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-3d10dc9c-76ee-4129-a7ed-0be292277391 admin admin] The token cache is being invalidate because project 6b50dc16d14d4b869f1a6555db6c8373 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86828) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 10:55:51.268714 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-3d10dc9c-76ee-4129-a7ed-0be292277391 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:55:51.269331 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-3d10dc9c-76ee-4129-a7ed-0be292277391 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:55:51.270850 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1327/2678] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Jul 27 10:55:51 2022] DELETE /identity/v3/projects/6b50dc16d14d4b869f1a6555db6c8373 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:55:56.716513 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff99f57a-6a06-42f3-9fc5-9c03fb96e376 admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:55:56.718008 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-ff99f57a-6a06-42f3-9fc5-9c03fb96e376 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:55:56.891691 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff99f57a-6a06-42f3-9fc5-9c03fb96e376 admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:55:56.945482 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9776d909-5be4-423a-be1c-f7dd04554020 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:55:56.946131 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9776d909-5be4-423a-be1c-f7dd04554020 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:55:56.946413 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9776d909-5be4-423a-be1c-f7dd04554020 admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:55:56.946719 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9776d909-5be4-423a-be1c-f7dd04554020 admin admin] PATH_INFO: `/v3/users/968007c31fde4b879da9b3ae9a1335aa` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:55:56.948468 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9776d909-5be4-423a-be1c-f7dd04554020 admin admin] RBAC: Authorizing `identity:delete_user(user_id=968007c31fde4b879da9b3ae9a1335aa)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:55:56.950370 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9776d909-5be4-423a-be1c-f7dd04554020 admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:55:56.986435 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:55:56.996446 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-9776d909-5be4-423a-be1c-f7dd04554020 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '968007c31fde4b879da9b3ae9a1335aa'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:55:57.003400 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-9776d909-5be4-423a-be1c-f7dd04554020 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '968007c31fde4b879da9b3ae9a1335aa'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:55:57.011204 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-9776d909-5be4-423a-be1c-f7dd04554020 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '968007c31fde4b879da9b3ae9a1335aa'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:55:57.011830 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-9776d909-5be4-423a-be1c-f7dd04554020 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '968007c31fde4b879da9b3ae9a1335aa'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:55:57.012439 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-9776d909-5be4-423a-be1c-f7dd04554020 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '968007c31fde4b879da9b3ae9a1335aa'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:55:57.029387 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1352/2679] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Jul 27 10:55:56 2022] DELETE /identity/v3/users/968007c31fde4b879da9b3ae9a1335aa => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:55:57.195898 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d10dc9c-76ee-4129-a7ed-0be292277391 admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:55:57.197495 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-3d10dc9c-76ee-4129-a7ed-0be292277391 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:55:57.210200 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d10dc9c-76ee-4129-a7ed-0be292277391 admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:55:57.268833 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c837eb-b330-44d7-a6ec-2653484e5d9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:55:57.269472 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c837eb-b330-44d7-a6ec-2653484e5d9e admin admin] REQUEST_METHOD: `DELETE` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:55:57.269752 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c837eb-b330-44d7-a6ec-2653484e5d9e admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:55:57.270026 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c837eb-b330-44d7-a6ec-2653484e5d9e admin admin] PATH_INFO: `/v3/projects/7b9629060ff841feaf09e03396ee02db` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:55:57.271191 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c837eb-b330-44d7-a6ec-2653484e5d9e admin admin] RBAC: Authorizing `identity:delete_project(project_id=7b9629060ff841feaf09e03396ee02db)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:55:57.273079 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c837eb-b330-44d7-a6ec-2653484e5d9e admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:55:57.320822 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-07c837eb-b330-44d7-a6ec-2653484e5d9e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7b9629060ff841feaf09e03396ee02db', 'request_id': 'req-07c837eb-b330-44d7-a6ec-2653484e5d9e'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:55:57.326851 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-07c837eb-b330-44d7-a6ec-2653484e5d9e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7b9629060ff841feaf09e03396ee02db', 'request_id': 'req-07c837eb-b330-44d7-a6ec-2653484e5d9e'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:55:57.333693 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-07c837eb-b330-44d7-a6ec-2653484e5d9e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7b9629060ff841feaf09e03396ee02db', 'request_id': 'req-07c837eb-b330-44d7-a6ec-2653484e5d9e'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:55:57.343683 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-07c837eb-b330-44d7-a6ec-2653484e5d9e admin admin] The token cache is being invalidate because project 7b9629060ff841feaf09e03396ee02db was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86828) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 10:55:57.343969 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-07c837eb-b330-44d7-a6ec-2653484e5d9e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:55:57.344590 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-07c837eb-b330-44d7-a6ec-2653484e5d9e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:55:57.346086 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1328/2680] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Jul 27 10:55:57 2022] DELETE /identity/v3/projects/7b9629060ff841feaf09e03396ee02db => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:55:57.350026 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9776d909-5be4-423a-be1c-f7dd04554020 admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:55:57.351529 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-9776d909-5be4-423a-be1c-f7dd04554020 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:55:57.365398 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9776d909-5be4-423a-be1c-f7dd04554020 admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:55:57.422443 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83936c98-1ca5-41be-8efe-23483989014b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:55:57.423033 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83936c98-1ca5-41be-8efe-23483989014b admin admin] REQUEST_METHOD: `DELETE` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:55:57.423312 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83936c98-1ca5-41be-8efe-23483989014b admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:55:57.423585 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83936c98-1ca5-41be-8efe-23483989014b admin admin] PATH_INFO: `/v3/users/61ef1abc462446d182358c3418d02b7b` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:55:57.424754 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83936c98-1ca5-41be-8efe-23483989014b admin admin] RBAC: Authorizing `identity:delete_user(user_id=61ef1abc462446d182358c3418d02b7b)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:55:57.426668 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83936c98-1ca5-41be-8efe-23483989014b admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:55:57.475181 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-83936c98-1ca5-41be-8efe-23483989014b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '61ef1abc462446d182358c3418d02b7b'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:55:57.482496 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-83936c98-1ca5-41be-8efe-23483989014b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '61ef1abc462446d182358c3418d02b7b'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:55:57.489960 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-83936c98-1ca5-41be-8efe-23483989014b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '61ef1abc462446d182358c3418d02b7b'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:55:57.490566 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-83936c98-1ca5-41be-8efe-23483989014b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '61ef1abc462446d182358c3418d02b7b'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:55:57.491519 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-83936c98-1ca5-41be-8efe-23483989014b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '61ef1abc462446d182358c3418d02b7b'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:55:57.508760 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1353/2681] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Jul 27 10:55:57 2022] DELETE /identity/v3/users/61ef1abc462446d182358c3418d02b7b => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:55:57.689592 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c837eb-b330-44d7-a6ec-2653484e5d9e admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:55:57.691039 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-07c837eb-b330-44d7-a6ec-2653484e5d9e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:55:57.703401 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c837eb-b330-44d7-a6ec-2653484e5d9e admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:55:57.707058 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:55:57.765129 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce2c962-93aa-442c-b384-7c6a700f4465 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:55:57.765768 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce2c962-93aa-442c-b384-7c6a700f4465 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:55:57.766047 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce2c962-93aa-442c-b384-7c6a700f4465 admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:55:57.766320 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce2c962-93aa-442c-b384-7c6a700f4465 admin admin] PATH_INFO: `/v3/projects/3768dc37162c436f8a446ddb54e121c2` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:55:57.767507 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce2c962-93aa-442c-b384-7c6a700f4465 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3768dc37162c436f8a446ddb54e121c2)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:55:57.769386 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce2c962-93aa-442c-b384-7c6a700f4465 admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:55:57.814513 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-2ce2c962-93aa-442c-b384-7c6a700f4465 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3768dc37162c436f8a446ddb54e121c2', 'request_id': 'req-2ce2c962-93aa-442c-b384-7c6a700f4465'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:55:57.820569 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-2ce2c962-93aa-442c-b384-7c6a700f4465 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3768dc37162c436f8a446ddb54e121c2', 'request_id': 'req-2ce2c962-93aa-442c-b384-7c6a700f4465'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:55:57.828442 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-2ce2c962-93aa-442c-b384-7c6a700f4465 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3768dc37162c436f8a446ddb54e121c2', 'request_id': 'req-2ce2c962-93aa-442c-b384-7c6a700f4465'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:55:57.845545 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-2ce2c962-93aa-442c-b384-7c6a700f4465 admin admin] The token cache is being invalidate because project 3768dc37162c436f8a446ddb54e121c2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86828) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 10:55:57.845840 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-2ce2c962-93aa-442c-b384-7c6a700f4465 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:55:57.846476 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-2ce2c962-93aa-442c-b384-7c6a700f4465 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:55:57.848009 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1329/2682] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Jul 27 10:55:57 2022] DELETE /identity/v3/projects/3768dc37162c436f8a446ddb54e121c2 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 10:55:57.853346 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41138b78-0963-48d1-8768-34fb8da06355 None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:55:57.853837 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41138b78-0963-48d1-8768-34fb8da06355 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:55:57.853992 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41138b78-0963-48d1-8768-34fb8da06355 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:55:57.902811 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:55:57.913509 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-41138b78-0963-48d1-8768-34fb8da06355 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:55:57.914940 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-41138b78-0963-48d1-8768-34fb8da06355 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:55:57.972395 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1354/2683] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:55:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:55:57.980220 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b90eaf2-7e94-41b6-b9dc-102ddd74009c None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:55:57.980509 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b90eaf2-7e94-41b6-b9dc-102ddd74009c None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:55:57.980786 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b90eaf2-7e94-41b6-b9dc-102ddd74009c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:55:57.999994 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-7b90eaf2-7e94-41b6-b9dc-102ddd74009c None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:55:58.001175 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-7b90eaf2-7e94-41b6-b9dc-102ddd74009c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:55:58.008793 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1330/2684] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:55:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 10:55:58.013305 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41138b78-0963-48d1-8768-34fb8da06355 None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:55:58.022784 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41138b78-0963-48d1-8768-34fb8da06355 None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:55:58.028096 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118119a8-9053-42b2-97d1-dc3da7845ff2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:55:58.028711 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118119a8-9053-42b2-97d1-dc3da7845ff2 admin admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:55:58.028995 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118119a8-9053-42b2-97d1-dc3da7845ff2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:55:58.029273 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118119a8-9053-42b2-97d1-dc3da7845ff2 admin admin] PATH_INFO: `/v3/domains` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:55:58.029971 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118119a8-9053-42b2-97d1-dc3da7845ff2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86827) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:55:58.030276 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118119a8-9053-42b2-97d1-dc3da7845ff2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:55:58.031967 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118119a8-9053-42b2-97d1-dc3da7845ff2 admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:55:58.043479 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1355/2685] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Jul 27 10:55:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:55:58.047506 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b90eaf2-7e94-41b6-b9dc-102ddd74009c None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:55:58.056527 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b90eaf2-7e94-41b6-b9dc-102ddd74009c None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:55:58.061181 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cd013f-86ec-40f8-b917-e35100a55243 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:55:58.061852 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cd013f-86ec-40f8-b917-e35100a55243 admin admin] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:55:58.062129 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cd013f-86ec-40f8-b917-e35100a55243 admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:55:58.062401 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cd013f-86ec-40f8-b917-e35100a55243 admin admin] PATH_INFO: `/v3/projects` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:55:58.063115 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91cd013f-86ec-40f8-b917-e35100a55243 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:55:58.065016 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91cd013f-86ec-40f8-b917-e35100a55243 admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:55:58.093282 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1331/2686] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Jul 27 10:55:58 2022] POST /identity/v3/projects => generated 375 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 10:55:58.097358 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118119a8-9053-42b2-97d1-dc3da7845ff2 admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:55:58.107280 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118119a8-9053-42b2-97d1-dc3da7845ff2 admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:55:58.163221 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55165622-402c-4e32-ba7f-b311aecab69b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:55:58.163833 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55165622-402c-4e32-ba7f-b311aecab69b admin admin] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:55:58.164115 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55165622-402c-4e32-ba7f-b311aecab69b admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:55:58.164391 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55165622-402c-4e32-ba7f-b311aecab69b admin admin] PATH_INFO: `/v3/users` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:55:58.165053 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55165622-402c-4e32-ba7f-b311aecab69b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:55:58.167028 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55165622-402c-4e32-ba7f-b311aecab69b admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:55:58.187278 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1356/2687] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Jul 27 10:55:58 2022] POST /identity/v3/users => generated 288 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 10:55:58.192067 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cd013f-86ec-40f8-b917-e35100a55243 admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:55:58.203036 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cd013f-86ec-40f8-b917-e35100a55243 admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:55:58.207597 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebda87a-c367-40e0-b959-5d65ef18af73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:55:58.208208 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebda87a-c367-40e0-b959-5d65ef18af73 admin admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:55:58.208490 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebda87a-c367-40e0-b959-5d65ef18af73 admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:55:58.208764 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebda87a-c367-40e0-b959-5d65ef18af73 admin admin] PATH_INFO: `/v3/roles` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:55:58.209456 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebda87a-c367-40e0-b959-5d65ef18af73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:55:58.211090 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebda87a-c367-40e0-b959-5d65ef18af73 admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:55:58.221220 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1332/2688] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 10:55:58 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 10:55:58.225023 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55165622-402c-4e32-ba7f-b311aecab69b admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:55:58.235036 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55165622-402c-4e32-ba7f-b311aecab69b admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:55:58.239655 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8a3fea-5e54-4e9e-9019-7b68200ec670 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:55:58.240259 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8a3fea-5e54-4e9e-9019-7b68200ec670 admin admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:55:58.240544 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8a3fea-5e54-4e9e-9019-7b68200ec670 admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:55:58.240822 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8a3fea-5e54-4e9e-9019-7b68200ec670 admin admin] PATH_INFO: `/v3/roles` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:55:58.241509 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8a3fea-5e54-4e9e-9019-7b68200ec670 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:55:58.243165 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8a3fea-5e54-4e9e-9019-7b68200ec670 admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:55:58.253616 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1357/2689] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 10:55:58 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 10:55:58.257559 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebda87a-c367-40e0-b959-5d65ef18af73 admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:55:58.268041 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebda87a-c367-40e0-b959-5d65ef18af73 admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:55:58.272643 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dbefa5-2656-4597-95f3-60669dc2032b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:55:58.273299 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dbefa5-2656-4597-95f3-60669dc2032b admin admin] REQUEST_METHOD: `PUT` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:55:58.273582 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dbefa5-2656-4597-95f3-60669dc2032b admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:55:58.273858 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dbefa5-2656-4597-95f3-60669dc2032b admin admin] PATH_INFO: `/v3/projects/85acc8e391c74237990d75e054b20f06/users/2d7d3a2c348146eeba95d9ef01e7c62b/roles/4c9fc0b3baed441da2126be41cc6b04a` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:55:58.288124 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7dbefa5-2656-4597-95f3-60669dc2032b admin admin] RBAC: Authorizing `identity:create_grant(project_id=85acc8e391c74237990d75e054b20f06, user_id=2d7d3a2c348146eeba95d9ef01e7c62b, role_id=4c9fc0b3baed441da2126be41cc6b04a)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:55:58.290783 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7dbefa5-2656-4597-95f3-60669dc2032b admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:55:58.303558 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1333/2690] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 10:55:58 2022] PUT /identity/v3/projects/85acc8e391c74237990d75e054b20f06/users/2d7d3a2c348146eeba95d9ef01e7c62b/roles/4c9fc0b3baed441da2126be41cc6b04a => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:55:58.315617 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac794fc-739c-4454-bf7d-b5463d79718d None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:55:58.315921 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac794fc-739c-4454-bf7d-b5463d79718d None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:55:58.316238 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac794fc-739c-4454-bf7d-b5463d79718d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:55:58.343836 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-9ac794fc-739c-4454-bf7d-b5463d79718d None None] MFA Rules not processed for user `2d7d3a2c348146eeba95d9ef01e7c62b`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:55:58.345039 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-9ac794fc-739c-4454-bf7d-b5463d79718d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:55:58.404275 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:55:58.410539 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1358/2691] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:55:58 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:55:58.421339 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dbefa5-2656-4597-95f3-60669dc2032b admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:55:58.423152 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-f7dbefa5-2656-4597-95f3-60669dc2032b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:55:58.460769 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:55:58.499339 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dbefa5-2656-4597-95f3-60669dc2032b admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:55:58.503085 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68b1bcb-fa96-49c0-8203-d9121b8554bb service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc03c5d232654878aa1ddce15f49fdaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:55:58.503703 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68b1bcb-fa96-49c0-8203-d9121b8554bb service glance] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:55:58.503985 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68b1bcb-fa96-49c0-8203-d9121b8554bb service glance] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:55:58.504260 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68b1bcb-fa96-49c0-8203-d9121b8554bb service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:55:58.512881 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68b1bcb-fa96-49c0-8203-d9121b8554bb service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:55:58.514298 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68b1bcb-fa96-49c0-8203-d9121b8554bb service glance] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:55:58.522931 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1334/2692] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:55:58 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 10:56:05.059087 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac794fc-739c-4454-bf7d-b5463d79718d None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:56:05.060506 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-9ac794fc-739c-4454-bf7d-b5463d79718d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:56:05.152802 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac794fc-739c-4454-bf7d-b5463d79718d None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:56:05.157786 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b261043e-4cba-4e6f-a31a-348c17b95193 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16cc6179e9a04b6398fbacb40ad1c555', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:56:05.158458 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b261043e-4cba-4e6f-a31a-348c17b95193 service placement] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:56:05.158784 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b261043e-4cba-4e6f-a31a-348c17b95193 service placement] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:56:05.159060 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b261043e-4cba-4e6f-a31a-348c17b95193 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:56:05.160823 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-b261043e-4cba-4e6f-a31a-348c17b95193 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:56:05.172641 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b261043e-4cba-4e6f-a31a-348c17b95193 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:56:05.174304 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b261043e-4cba-4e6f-a31a-348c17b95193 service placement] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:56:05.184611 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1359/2693] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:56:05 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 10:56:08.519007 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=18 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:56:13.999747 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68b1bcb-fa96-49c0-8203-d9121b8554bb service glance] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:56:14.060489 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68b1bcb-fa96-49c0-8203-d9121b8554bb service glance] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:56:14.065007 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfaf62a-552a-43fa-8e4e-578ea7be4fbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:56:14.065634 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfaf62a-552a-43fa-8e4e-578ea7be4fbd admin admin] REQUEST_METHOD: `DELETE` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:56:14.065915 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfaf62a-552a-43fa-8e4e-578ea7be4fbd admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:56:14.066198 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfaf62a-552a-43fa-8e4e-578ea7be4fbd admin admin] PATH_INFO: `/v3/users/2d7d3a2c348146eeba95d9ef01e7c62b` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:56:14.067366 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bfaf62a-552a-43fa-8e4e-578ea7be4fbd admin admin] RBAC: Authorizing `identity:delete_user(user_id=2d7d3a2c348146eeba95d9ef01e7c62b)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:56:14.069263 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bfaf62a-552a-43fa-8e4e-578ea7be4fbd admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:56:14.114934 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-1bfaf62a-552a-43fa-8e4e-578ea7be4fbd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2d7d3a2c348146eeba95d9ef01e7c62b'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:56:14.121639 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-1bfaf62a-552a-43fa-8e4e-578ea7be4fbd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2d7d3a2c348146eeba95d9ef01e7c62b'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:56:14.128643 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-1bfaf62a-552a-43fa-8e4e-578ea7be4fbd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2d7d3a2c348146eeba95d9ef01e7c62b'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:56:14.129549 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-1bfaf62a-552a-43fa-8e4e-578ea7be4fbd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2d7d3a2c348146eeba95d9ef01e7c62b'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:56:14.130167 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-1bfaf62a-552a-43fa-8e4e-578ea7be4fbd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2d7d3a2c348146eeba95d9ef01e7c62b'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:56:14.147329 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1335/2694] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Jul 27 10:56:13 2022] DELETE /identity/v3/users/2d7d3a2c348146eeba95d9ef01e7c62b => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:56:14.152273 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7952e70d-ddb8-4d99-960e-7e205c0231db None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:56:14.152748 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7952e70d-ddb8-4d99-960e-7e205c0231db None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:56:14.152866 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7952e70d-ddb8-4d99-960e-7e205c0231db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:56:14.171699 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-7952e70d-ddb8-4d99-960e-7e205c0231db None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:56:14.173240 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-7952e70d-ddb8-4d99-960e-7e205c0231db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:56:14.230298 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1360/2695] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:56:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:56:14.239959 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfaf62a-552a-43fa-8e4e-578ea7be4fbd admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:56:14.241355 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-1bfaf62a-552a-43fa-8e4e-578ea7be4fbd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:56:14.321670 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfaf62a-552a-43fa-8e4e-578ea7be4fbd admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:56:14.325888 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be635f4f-3155-4b9e-b234-95febe5d1cad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7922d0a1a7c4e808d0b2a80dca0da40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:56:14.326505 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be635f4f-3155-4b9e-b234-95febe5d1cad service neutron] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:56:14.326944 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be635f4f-3155-4b9e-b234-95febe5d1cad service neutron] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:56:14.327223 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be635f4f-3155-4b9e-b234-95febe5d1cad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:56:14.336736 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be635f4f-3155-4b9e-b234-95febe5d1cad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:56:14.338169 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be635f4f-3155-4b9e-b234-95febe5d1cad service neutron] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:56:14.349652 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1336/2696] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:56:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 10:56:14.616861 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7952e70d-ddb8-4d99-960e-7e205c0231db None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:56:14.618376 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-7952e70d-ddb8-4d99-960e-7e205c0231db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:56:14.631941 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7952e70d-ddb8-4d99-960e-7e205c0231db None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:56:14.638568 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c589a256-393b-4786-ae0a-4e258e777360 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:56:14.639194 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c589a256-393b-4786-ae0a-4e258e777360 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:56:14.639478 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c589a256-393b-4786-ae0a-4e258e777360 admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:56:14.639852 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c589a256-393b-4786-ae0a-4e258e777360 admin admin] PATH_INFO: `/v3/projects/85acc8e391c74237990d75e054b20f06` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:56:14.641619 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c589a256-393b-4786-ae0a-4e258e777360 admin admin] RBAC: Authorizing `identity:delete_project(project_id=85acc8e391c74237990d75e054b20f06)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:56:14.643830 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c589a256-393b-4786-ae0a-4e258e777360 admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:56:14.697292 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-c589a256-393b-4786-ae0a-4e258e777360 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '85acc8e391c74237990d75e054b20f06', 'request_id': 'req-c589a256-393b-4786-ae0a-4e258e777360'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:56:14.706889 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-c589a256-393b-4786-ae0a-4e258e777360 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '85acc8e391c74237990d75e054b20f06', 'request_id': 'req-c589a256-393b-4786-ae0a-4e258e777360'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:56:14.716022 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-c589a256-393b-4786-ae0a-4e258e777360 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '85acc8e391c74237990d75e054b20f06', 'request_id': 'req-c589a256-393b-4786-ae0a-4e258e777360'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:56:14.719251 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:56:14.730330 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-c589a256-393b-4786-ae0a-4e258e777360 admin admin] The token cache is being invalidate because project 85acc8e391c74237990d75e054b20f06 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86827) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 10:56:14.730648 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-c589a256-393b-4786-ae0a-4e258e777360 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:56:14.731280 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-c589a256-393b-4786-ae0a-4e258e777360 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:56:14.732787 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1361/2697] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Jul 27 10:56:14 2022] DELETE /identity/v3/projects/85acc8e391c74237990d75e054b20f06 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:56:14.737968 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cce43ee-36b5-42ee-b7d7-97557810406a None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:56:14.738258 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cce43ee-36b5-42ee-b7d7-97557810406a None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:56:14.738539 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cce43ee-36b5-42ee-b7d7-97557810406a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:56:14.746815 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:56:14.761441 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-3cce43ee-36b5-42ee-b7d7-97557810406a None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:56:14.762795 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-3cce43ee-36b5-42ee-b7d7-97557810406a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:56:14.824065 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1337/2698] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:56:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:56:14.831094 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbfc53b-8eec-4622-ac8a-345e9a24c9da None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:56:14.831411 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbfc53b-8eec-4622-ac8a-345e9a24c9da None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:56:14.831687 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbfc53b-8eec-4622-ac8a-345e9a24c9da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:56:14.851524 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-9dbfc53b-8eec-4622-ac8a-345e9a24c9da None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:56:14.852883 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-9dbfc53b-8eec-4622-ac8a-345e9a24c9da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:56:14.860872 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1362/2699] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:56:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:56:14.865005 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cce43ee-36b5-42ee-b7d7-97557810406a None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:56:14.876103 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cce43ee-36b5-42ee-b7d7-97557810406a None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:56:14.881309 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba24b1cd-0dcb-48dc-be07-24480b49adad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:56:14.882077 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba24b1cd-0dcb-48dc-be07-24480b49adad admin admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:56:14.882362 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba24b1cd-0dcb-48dc-be07-24480b49adad admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:56:14.882678 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba24b1cd-0dcb-48dc-be07-24480b49adad admin admin] PATH_INFO: `/v3/domains` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:56:14.883380 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba24b1cd-0dcb-48dc-be07-24480b49adad admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86828) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:56:14.883752 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba24b1cd-0dcb-48dc-be07-24480b49adad admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:56:14.885292 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba24b1cd-0dcb-48dc-be07-24480b49adad admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:56:14.898078 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1338/2700] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Jul 27 10:56:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:56:14.902533 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbfc53b-8eec-4622-ac8a-345e9a24c9da None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:56:14.911994 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbfc53b-8eec-4622-ac8a-345e9a24c9da None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:56:14.916886 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b047733a-18e6-40a0-b4a4-8d585255a65d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:56:14.917499 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b047733a-18e6-40a0-b4a4-8d585255a65d admin admin] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:56:14.917781 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b047733a-18e6-40a0-b4a4-8d585255a65d admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:56:14.918056 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b047733a-18e6-40a0-b4a4-8d585255a65d admin admin] PATH_INFO: `/v3/projects` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:56:14.918819 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b047733a-18e6-40a0-b4a4-8d585255a65d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:56:14.920932 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b047733a-18e6-40a0-b4a4-8d585255a65d admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:56:14.957369 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1363/2701] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Jul 27 10:56:14 2022] POST /identity/v3/projects => generated 371 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 10:56:14.958657 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba24b1cd-0dcb-48dc-be07-24480b49adad admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:56:14.969001 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba24b1cd-0dcb-48dc-be07-24480b49adad admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:56:15.041148 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45867a01-57eb-4900-aea6-87434fff3c1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:56:15.042301 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45867a01-57eb-4900-aea6-87434fff3c1f admin admin] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:56:15.042579 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45867a01-57eb-4900-aea6-87434fff3c1f admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:56:15.050218 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45867a01-57eb-4900-aea6-87434fff3c1f admin admin] PATH_INFO: `/v3/users` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:56:15.050218 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45867a01-57eb-4900-aea6-87434fff3c1f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:56:15.050218 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45867a01-57eb-4900-aea6-87434fff3c1f admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:56:15.076125 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1339/2702] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Jul 27 10:56:14 2022] POST /identity/v3/users => generated 286 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 10:56:15.077997 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b047733a-18e6-40a0-b4a4-8d585255a65d admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:56:15.091875 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b047733a-18e6-40a0-b4a4-8d585255a65d admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:56:15.096843 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc45c14-0370-420f-a7d6-0471195824a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:56:15.097479 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc45c14-0370-420f-a7d6-0471195824a9 admin admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:56:15.097767 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc45c14-0370-420f-a7d6-0471195824a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:56:15.098046 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc45c14-0370-420f-a7d6-0471195824a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:56:15.098821 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc45c14-0370-420f-a7d6-0471195824a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:56:15.100451 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc45c14-0370-420f-a7d6-0471195824a9 admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:56:15.112098 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1364/2703] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 10:56:15 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 10:56:15.116354 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45867a01-57eb-4900-aea6-87434fff3c1f admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:56:15.128049 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45867a01-57eb-4900-aea6-87434fff3c1f admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:56:15.132943 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f5a21a-e573-4723-bd67-4319a6e1f45e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:56:15.133586 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f5a21a-e573-4723-bd67-4319a6e1f45e admin admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:56:15.133867 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f5a21a-e573-4723-bd67-4319a6e1f45e admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:56:15.134144 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f5a21a-e573-4723-bd67-4319a6e1f45e admin admin] PATH_INFO: `/v3/roles` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:56:15.134893 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f5a21a-e573-4723-bd67-4319a6e1f45e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:56:15.136509 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f5a21a-e573-4723-bd67-4319a6e1f45e admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:56:15.147965 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1340/2704] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 10:56:15 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 10:56:15.152353 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc45c14-0370-420f-a7d6-0471195824a9 admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:56:15.162718 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc45c14-0370-420f-a7d6-0471195824a9 admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:56:15.168101 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee20d2e-8e8c-44e8-ac80-924f5bc460e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:56:15.168789 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee20d2e-8e8c-44e8-ac80-924f5bc460e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:56:15.169070 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee20d2e-8e8c-44e8-ac80-924f5bc460e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:56:15.169348 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee20d2e-8e8c-44e8-ac80-924f5bc460e0 admin admin] PATH_INFO: `/v3/projects/bcafd47e5c5648c2a7f6da78c40fa7e1/users/ab8802a9291f43aca2b24d5ba772f260/roles/4c9fc0b3baed441da2126be41cc6b04a` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:56:15.186157 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee20d2e-8e8c-44e8-ac80-924f5bc460e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bcafd47e5c5648c2a7f6da78c40fa7e1, user_id=ab8802a9291f43aca2b24d5ba772f260, role_id=4c9fc0b3baed441da2126be41cc6b04a)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:56:15.188866 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee20d2e-8e8c-44e8-ac80-924f5bc460e0 admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:56:15.203068 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1365/2705] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 10:56:15 2022] PUT /identity/v3/projects/bcafd47e5c5648c2a7f6da78c40fa7e1/users/ab8802a9291f43aca2b24d5ba772f260/roles/4c9fc0b3baed441da2126be41cc6b04a => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:56:15.208360 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b755bb3c-f103-43a4-b6fc-97fb4191d037 None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:56:15.208714 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b755bb3c-f103-43a4-b6fc-97fb4191d037 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:56:15.208997 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b755bb3c-f103-43a4-b6fc-97fb4191d037 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:56:15.228585 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-b755bb3c-f103-43a4-b6fc-97fb4191d037 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:56:15.230507 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-b755bb3c-f103-43a4-b6fc-97fb4191d037 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:56:15.298812 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1341/2706] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:56:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:56:15.308840 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee20d2e-8e8c-44e8-ac80-924f5bc460e0 admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:56:15.310593 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-1ee20d2e-8e8c-44e8-ac80-924f5bc460e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:56:15.371975 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee20d2e-8e8c-44e8-ac80-924f5bc460e0 admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:56:15.375969 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b58b7f-07d1-44ea-9394-d36300abd1b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7922d0a1a7c4e808d0b2a80dca0da40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:56:15.376700 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b58b7f-07d1-44ea-9394-d36300abd1b0 service neutron] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:56:15.377041 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b58b7f-07d1-44ea-9394-d36300abd1b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:56:15.377328 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b58b7f-07d1-44ea-9394-d36300abd1b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:56:15.388037 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b58b7f-07d1-44ea-9394-d36300abd1b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:56:15.389473 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b58b7f-07d1-44ea-9394-d36300abd1b0 service neutron] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:56:15.403544 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1366/2707] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:56:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 10:56:16.027196 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9622db59-f261-49db-b020-5d5184479755 None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:56:16.027492 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9622db59-f261-49db-b020-5d5184479755 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:56:16.027776 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9622db59-f261-49db-b020-5d5184479755 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:56:16.048493 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:56:16.060429 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-9622db59-f261-49db-b020-5d5184479755 None None] MFA Rules not processed for user `ab8802a9291f43aca2b24d5ba772f260`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:56:16.061645 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-9622db59-f261-49db-b020-5d5184479755 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:56:16.127718 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1342/2708] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:56:16 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:56:16.138699 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b58b7f-07d1-44ea-9394-d36300abd1b0 service neutron] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:56:16.140106 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-e8b58b7f-07d1-44ea-9394-d36300abd1b0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:56:16.218542 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b58b7f-07d1-44ea-9394-d36300abd1b0 service neutron] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:56:16.222221 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb39981-2937-4b3b-bc4f-0fe88dd1fb71 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '864d35f268cf45148756109759bb2743', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:56:16.222868 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb39981-2937-4b3b-bc4f-0fe88dd1fb71 service cinder] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:56:16.223155 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb39981-2937-4b3b-bc4f-0fe88dd1fb71 service cinder] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:56:16.223432 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb39981-2937-4b3b-bc4f-0fe88dd1fb71 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:56:16.232389 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb39981-2937-4b3b-bc4f-0fe88dd1fb71 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:56:16.233813 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb39981-2937-4b3b-bc4f-0fe88dd1fb71 service cinder] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:56:16.242568 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1367/2709] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:56:16 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 10:56:17.622079 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06aa9b0-ce0b-4cdf-8ab0-80650184c37c None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:56:17.622927 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06aa9b0-ce0b-4cdf-8ab0-80650184c37c None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:56:17.623051 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06aa9b0-ce0b-4cdf-8ab0-80650184c37c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:56:17.650026 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-a06aa9b0-ce0b-4cdf-8ab0-80650184c37c None None] MFA Rules not processed for user `ab8802a9291f43aca2b24d5ba772f260`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:56:17.651117 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-a06aa9b0-ce0b-4cdf-8ab0-80650184c37c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:56:17.660380 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1343/2710] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:56:17 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:56:26.238727 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=91,keystone:DELETE=1,keystone:INSERT=2 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:56:27.561411 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02485849-07fd-4ae2-9c71-8d50776446cf None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:56:27.562060 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02485849-07fd-4ae2-9c71-8d50776446cf None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:56:27.562060 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02485849-07fd-4ae2-9c71-8d50776446cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:56:27.582990 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-02485849-07fd-4ae2-9c71-8d50776446cf None None] MFA Rules not processed for user `afd04d5fdc7e44c89edddf97be489db9`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:56:27.584157 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-02485849-07fd-4ae2-9c71-8d50776446cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:56:27.639965 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=31 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:56:27.647228 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1368/2711] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:56:27 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:56:33.188383 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d38ede8-8c6f-4949-8100-5d2fb2dab76e None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:56:33.188882 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d38ede8-8c6f-4949-8100-5d2fb2dab76e None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:56:33.189019 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d38ede8-8c6f-4949-8100-5d2fb2dab76e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:56:33.208167 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-0d38ede8-8c6f-4949-8100-5d2fb2dab76e None None] MFA Rules not processed for user `ab8802a9291f43aca2b24d5ba772f260`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:56:33.209362 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-0d38ede8-8c6f-4949-8100-5d2fb2dab76e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:56:33.216944 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1344/2712] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:56:33 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:56:37.634532 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=25 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:56:43.203001 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=3 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:56:49.332553 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3744bf83-cd9c-45e2-b529-cdb459f8b5bf None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:56:49.333033 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3744bf83-cd9c-45e2-b529-cdb459f8b5bf None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:56:49.333132 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3744bf83-cd9c-45e2-b529-cdb459f8b5bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:56:49.361335 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-3744bf83-cd9c-45e2-b529-cdb459f8b5bf None None] MFA Rules not processed for user `3d9fed24b4ba44b0b5961a603c4362c4`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:56:49.362534 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-3744bf83-cd9c-45e2-b529-cdb459f8b5bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:56:49.424101 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1369/2713] 10.222.0.36 () {58 vars in 1009 bytes} [Wed Jul 27 10:56:49 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:56:49.469371 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d38ede8-8c6f-4949-8100-5d2fb2dab76e None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:56:49.470815 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-0d38ede8-8c6f-4949-8100-5d2fb2dab76e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:56:49.538493 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d38ede8-8c6f-4949-8100-5d2fb2dab76e None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:56:49.543992 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49c2350-cb74-43d6-9e1d-f06ff05be188 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89ac506eefcd4e42a84b44fa15fb3975', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:56:49.544616 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49c2350-cb74-43d6-9e1d-f06ff05be188 service swift] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:56:49.544901 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49c2350-cb74-43d6-9e1d-f06ff05be188 service swift] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:56:49.545175 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49c2350-cb74-43d6-9e1d-f06ff05be188 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:56:49.553272 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49c2350-cb74-43d6-9e1d-f06ff05be188 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:56:49.554880 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49c2350-cb74-43d6-9e1d-f06ff05be188 service swift] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:56:49.563421 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1345/2714] 10.222.0.36 () {60 vars in 1384 bytes} [Wed Jul 27 10:56:49 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 27 10:56:59.417097 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=32 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:56:59.560262 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=32 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:57:04.222810 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3956ea-77f2-4f20-abf9-33f1fe183806 None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:57:04.223292 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3956ea-77f2-4f20-abf9-33f1fe183806 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:57:04.223405 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3956ea-77f2-4f20-abf9-33f1fe183806 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:57:04.242747 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-7a3956ea-77f2-4f20-abf9-33f1fe183806 None None] MFA Rules not processed for user `3d9fed24b4ba44b0b5961a603c4362c4`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:57:04.243967 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-7a3956ea-77f2-4f20-abf9-33f1fe183806 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:57:04.251682 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1370/2715] 10.222.0.36 () {58 vars in 967 bytes} [Wed Jul 27 10:57:04 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:57:04.263596 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49c2350-cb74-43d6-9e1d-f06ff05be188 service swift] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:57:04.274029 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49c2350-cb74-43d6-9e1d-f06ff05be188 service swift] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:57:04.279789 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67fc6b6-f052-4ad7-8d06-8bd172cba235 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89ac506eefcd4e42a84b44fa15fb3975', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:57:04.280403 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67fc6b6-f052-4ad7-8d06-8bd172cba235 service swift] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:57:04.280683 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67fc6b6-f052-4ad7-8d06-8bd172cba235 service swift] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:57:04.280954 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67fc6b6-f052-4ad7-8d06-8bd172cba235 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:57:04.288878 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67fc6b6-f052-4ad7-8d06-8bd172cba235 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:57:04.290453 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67fc6b6-f052-4ad7-8d06-8bd172cba235 service swift] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:57:04.298837 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1346/2716] 10.222.0.36 () {60 vars in 1384 bytes} [Wed Jul 27 10:57:04 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 27 10:57:06.197074 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3956ea-77f2-4f20-abf9-33f1fe183806 None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:57:06.209651 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3956ea-77f2-4f20-abf9-33f1fe183806 None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:57:06.214321 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172abfc1-d434-407a-9cdf-d32b027d6765 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:57:06.215768 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172abfc1-d434-407a-9cdf-d32b027d6765 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:57:06.216061 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172abfc1-d434-407a-9cdf-d32b027d6765 admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:57:06.216327 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172abfc1-d434-407a-9cdf-d32b027d6765 admin admin] PATH_INFO: `/v3/users/ab8802a9291f43aca2b24d5ba772f260` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:57:06.217509 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172abfc1-d434-407a-9cdf-d32b027d6765 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ab8802a9291f43aca2b24d5ba772f260)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:57:06.219473 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172abfc1-d434-407a-9cdf-d32b027d6765 admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:57:06.265298 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-172abfc1-d434-407a-9cdf-d32b027d6765 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ab8802a9291f43aca2b24d5ba772f260'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:57:06.272149 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-172abfc1-d434-407a-9cdf-d32b027d6765 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ab8802a9291f43aca2b24d5ba772f260'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:57:06.279640 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-172abfc1-d434-407a-9cdf-d32b027d6765 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ab8802a9291f43aca2b24d5ba772f260'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:57:06.280273 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-172abfc1-d434-407a-9cdf-d32b027d6765 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ab8802a9291f43aca2b24d5ba772f260'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:57:06.280896 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-172abfc1-d434-407a-9cdf-d32b027d6765 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ab8802a9291f43aca2b24d5ba772f260'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:57:06.298826 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1371/2717] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Jul 27 10:57:06 2022] DELETE /identity/v3/users/ab8802a9291f43aca2b24d5ba772f260 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:57:06.471541 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67fc6b6-f052-4ad7-8d06-8bd172cba235 service swift] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:57:06.482373 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67fc6b6-f052-4ad7-8d06-8bd172cba235 service swift] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:57:06.539741 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a6c4c8-fb5d-484f-a07a-08ad90fdedbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:57:06.540384 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a6c4c8-fb5d-484f-a07a-08ad90fdedbf admin admin] REQUEST_METHOD: `DELETE` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:57:06.540750 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a6c4c8-fb5d-484f-a07a-08ad90fdedbf admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:57:06.541028 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a6c4c8-fb5d-484f-a07a-08ad90fdedbf admin admin] PATH_INFO: `/v3/projects/bcafd47e5c5648c2a7f6da78c40fa7e1` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:57:06.542184 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a6c4c8-fb5d-484f-a07a-08ad90fdedbf admin admin] RBAC: Authorizing `identity:delete_project(project_id=bcafd47e5c5648c2a7f6da78c40fa7e1)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:57:06.544113 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a6c4c8-fb5d-484f-a07a-08ad90fdedbf admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:57:06.589420 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-78a6c4c8-fb5d-484f-a07a-08ad90fdedbf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bcafd47e5c5648c2a7f6da78c40fa7e1', 'request_id': 'req-78a6c4c8-fb5d-484f-a07a-08ad90fdedbf'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:57:06.595447 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-78a6c4c8-fb5d-484f-a07a-08ad90fdedbf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bcafd47e5c5648c2a7f6da78c40fa7e1', 'request_id': 'req-78a6c4c8-fb5d-484f-a07a-08ad90fdedbf'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:57:06.602539 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-78a6c4c8-fb5d-484f-a07a-08ad90fdedbf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bcafd47e5c5648c2a7f6da78c40fa7e1', 'request_id': 'req-78a6c4c8-fb5d-484f-a07a-08ad90fdedbf'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:57:06.613336 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-78a6c4c8-fb5d-484f-a07a-08ad90fdedbf admin admin] The token cache is being invalidate because project bcafd47e5c5648c2a7f6da78c40fa7e1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86828) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 10:57:06.613683 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-78a6c4c8-fb5d-484f-a07a-08ad90fdedbf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:57:06.614379 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-78a6c4c8-fb5d-484f-a07a-08ad90fdedbf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:57:06.615923 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1347/2718] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Jul 27 10:57:06 2022] DELETE /identity/v3/projects/bcafd47e5c5648c2a7f6da78c40fa7e1 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:57:06.620858 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66a16b4-bf76-44ef-af3b-59f14d147be0 None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:57:06.621637 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66a16b4-bf76-44ef-af3b-59f14d147be0 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:57:06.621941 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66a16b4-bf76-44ef-af3b-59f14d147be0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:57:06.642300 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-d66a16b4-bf76-44ef-af3b-59f14d147be0 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:57:06.643544 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-d66a16b4-bf76-44ef-af3b-59f14d147be0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:57:06.703936 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1372/2719] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:57:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:57:06.711673 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24501026-1d55-4bd7-bf36-d405e0283875 None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:57:06.711958 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24501026-1d55-4bd7-bf36-d405e0283875 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:57:06.712263 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24501026-1d55-4bd7-bf36-d405e0283875 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:57:06.733151 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-24501026-1d55-4bd7-bf36-d405e0283875 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:57:06.734280 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-24501026-1d55-4bd7-bf36-d405e0283875 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:57:06.742206 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1348/2720] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:57:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:57:06.747322 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66a16b4-bf76-44ef-af3b-59f14d147be0 None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:57:06.756892 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66a16b4-bf76-44ef-af3b-59f14d147be0 None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:57:06.761339 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ae212e-1e85-400b-b820-bb8160d77fc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:57:06.761965 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ae212e-1e85-400b-b820-bb8160d77fc7 admin admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:57:06.762243 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ae212e-1e85-400b-b820-bb8160d77fc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:57:06.762513 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ae212e-1e85-400b-b820-bb8160d77fc7 admin admin] PATH_INFO: `/v3/domains` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:57:06.763237 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ae212e-1e85-400b-b820-bb8160d77fc7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86827) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:57:06.763537 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ae212e-1e85-400b-b820-bb8160d77fc7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:57:06.765126 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ae212e-1e85-400b-b820-bb8160d77fc7 admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:57:06.776318 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1373/2721] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Jul 27 10:57:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:57:06.780412 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24501026-1d55-4bd7-bf36-d405e0283875 None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:57:06.789667 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24501026-1d55-4bd7-bf36-d405e0283875 None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:57:06.794995 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fa3cd1-6fe1-4030-85d7-1604c51b3f36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:57:06.795604 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fa3cd1-6fe1-4030-85d7-1604c51b3f36 admin admin] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:57:06.795892 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fa3cd1-6fe1-4030-85d7-1604c51b3f36 admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:57:06.796170 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fa3cd1-6fe1-4030-85d7-1604c51b3f36 admin admin] PATH_INFO: `/v3/projects` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:57:06.796871 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fa3cd1-6fe1-4030-85d7-1604c51b3f36 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:57:06.798792 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fa3cd1-6fe1-4030-85d7-1604c51b3f36 admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:57:06.825832 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1349/2722] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Jul 27 10:57:06 2022] POST /identity/v3/projects => generated 375 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 10:57:06.830192 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ae212e-1e85-400b-b820-bb8160d77fc7 admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:57:06.840257 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ae212e-1e85-400b-b820-bb8160d77fc7 admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:57:06.887737 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:57:06.898068 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b53d4a-ceae-4156-9fdd-8045171e8479 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:57:06.898721 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b53d4a-ceae-4156-9fdd-8045171e8479 admin admin] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:57:06.899011 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b53d4a-ceae-4156-9fdd-8045171e8479 admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:57:06.899290 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b53d4a-ceae-4156-9fdd-8045171e8479 admin admin] PATH_INFO: `/v3/users` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:57:06.899948 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b53d4a-ceae-4156-9fdd-8045171e8479 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:57:06.901961 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b53d4a-ceae-4156-9fdd-8045171e8479 admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:57:06.926919 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1374/2723] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Jul 27 10:57:06 2022] POST /identity/v3/users => generated 288 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 10:57:06.931338 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fa3cd1-6fe1-4030-85d7-1604c51b3f36 admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:57:06.942766 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fa3cd1-6fe1-4030-85d7-1604c51b3f36 admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:57:06.947392 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7077f2-46e6-40f7-904d-2db9a614adcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:57:06.948011 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7077f2-46e6-40f7-904d-2db9a614adcd admin admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:57:06.948292 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7077f2-46e6-40f7-904d-2db9a614adcd admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:57:06.948567 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7077f2-46e6-40f7-904d-2db9a614adcd admin admin] PATH_INFO: `/v3/roles` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:57:06.949262 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7077f2-46e6-40f7-904d-2db9a614adcd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:57:06.950900 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7077f2-46e6-40f7-904d-2db9a614adcd admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:57:06.961002 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1350/2724] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 10:57:06 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 10:57:06.964940 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b53d4a-ceae-4156-9fdd-8045171e8479 admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:57:06.974910 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b53d4a-ceae-4156-9fdd-8045171e8479 admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:57:06.979524 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e1babc-4e19-4846-a556-5b09d3baab51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:57:06.981382 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e1babc-4e19-4846-a556-5b09d3baab51 admin admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:57:06.981382 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e1babc-4e19-4846-a556-5b09d3baab51 admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:57:06.981382 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e1babc-4e19-4846-a556-5b09d3baab51 admin admin] PATH_INFO: `/v3/roles` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:57:06.981862 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e1babc-4e19-4846-a556-5b09d3baab51 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:57:06.983776 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e1babc-4e19-4846-a556-5b09d3baab51 admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:57:06.995118 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1375/2725] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 10:57:06 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 10:57:07.000044 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7077f2-46e6-40f7-904d-2db9a614adcd admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:57:07.010039 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7077f2-46e6-40f7-904d-2db9a614adcd admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:57:07.014699 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0803f53d-2eb8-450d-8533-4fa8a785d7da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:57:07.015360 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0803f53d-2eb8-450d-8533-4fa8a785d7da admin admin] REQUEST_METHOD: `PUT` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:57:07.015644 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0803f53d-2eb8-450d-8533-4fa8a785d7da admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:57:07.015926 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0803f53d-2eb8-450d-8533-4fa8a785d7da admin admin] PATH_INFO: `/v3/projects/3074671803cb4acdbab557c073a92f57/users/50b3776584a440939bcdb7ba592a86eb/roles/4c9fc0b3baed441da2126be41cc6b04a` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:57:07.035849 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0803f53d-2eb8-450d-8533-4fa8a785d7da admin admin] RBAC: Authorizing `identity:create_grant(project_id=3074671803cb4acdbab557c073a92f57, user_id=50b3776584a440939bcdb7ba592a86eb, role_id=4c9fc0b3baed441da2126be41cc6b04a)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:57:07.038453 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0803f53d-2eb8-450d-8533-4fa8a785d7da admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:57:07.053263 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1351/2726] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 10:57:06 2022] PUT /identity/v3/projects/3074671803cb4acdbab557c073a92f57/users/50b3776584a440939bcdb7ba592a86eb/roles/4c9fc0b3baed441da2126be41cc6b04a => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:57:07.064250 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337d5da4-cc27-48a4-a2d6-36c7880bd56a None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:57:07.064543 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337d5da4-cc27-48a4-a2d6-36c7880bd56a None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:57:07.064823 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337d5da4-cc27-48a4-a2d6-36c7880bd56a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:57:07.098136 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-337d5da4-cc27-48a4-a2d6-36c7880bd56a None None] MFA Rules not processed for user `50b3776584a440939bcdb7ba592a86eb`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:57:07.099420 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-337d5da4-cc27-48a4-a2d6-36c7880bd56a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:57:07.163822 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1376/2727] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:57:07 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:57:07.175255 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0803f53d-2eb8-450d-8533-4fa8a785d7da admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:57:07.176935 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-0803f53d-2eb8-450d-8533-4fa8a785d7da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:57:07.191373 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:57:07.236273 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0803f53d-2eb8-450d-8533-4fa8a785d7da admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:57:07.240150 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d30fe1-d8b1-4070-9887-ac91cabe5a87 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '864d35f268cf45148756109759bb2743', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:57:07.240769 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d30fe1-d8b1-4070-9887-ac91cabe5a87 service cinder] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:57:07.241058 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d30fe1-d8b1-4070-9887-ac91cabe5a87 service cinder] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:57:07.241343 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d30fe1-d8b1-4070-9887-ac91cabe5a87 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:57:07.250363 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d30fe1-d8b1-4070-9887-ac91cabe5a87 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:57:07.251836 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d30fe1-d8b1-4070-9887-ac91cabe5a87 service cinder] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:57:07.260673 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1352/2728] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:57:07 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 10:57:17.067783 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337d5da4-cc27-48a4-a2d6-36c7880bd56a None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:57:17.140300 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337d5da4-cc27-48a4-a2d6-36c7880bd56a None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:57:17.146191 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1a506d-1b41-4384-b359-6526150a77d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:57:17.146869 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1a506d-1b41-4384-b359-6526150a77d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:57:17.147160 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1a506d-1b41-4384-b359-6526150a77d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:57:17.147440 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1a506d-1b41-4384-b359-6526150a77d5 admin admin] PATH_INFO: `/v3/users/50b3776584a440939bcdb7ba592a86eb` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:57:17.148639 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1a506d-1b41-4384-b359-6526150a77d5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=50b3776584a440939bcdb7ba592a86eb)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:57:17.150561 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1a506d-1b41-4384-b359-6526150a77d5 admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:57:17.197351 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-8d1a506d-1b41-4384-b359-6526150a77d5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '50b3776584a440939bcdb7ba592a86eb'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:57:17.204969 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-8d1a506d-1b41-4384-b359-6526150a77d5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '50b3776584a440939bcdb7ba592a86eb'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:57:17.212369 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-8d1a506d-1b41-4384-b359-6526150a77d5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '50b3776584a440939bcdb7ba592a86eb'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:57:17.213043 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-8d1a506d-1b41-4384-b359-6526150a77d5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '50b3776584a440939bcdb7ba592a86eb'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:57:17.213732 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-8d1a506d-1b41-4384-b359-6526150a77d5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '50b3776584a440939bcdb7ba592a86eb'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:57:17.214920 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:57:17.233478 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1377/2729] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Jul 27 10:57:17 2022] DELETE /identity/v3/users/50b3776584a440939bcdb7ba592a86eb => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:57:17.237989 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e75f297-9ea6-44c8-bf1e-0cd4c7d39eab None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:57:17.238469 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e75f297-9ea6-44c8-bf1e-0cd4c7d39eab None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:57:17.238588 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e75f297-9ea6-44c8-bf1e-0cd4c7d39eab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:57:17.257463 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-8e75f297-9ea6-44c8-bf1e-0cd4c7d39eab None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:57:17.258685 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-8e75f297-9ea6-44c8-bf1e-0cd4c7d39eab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:57:17.315852 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1353/2730] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:57:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:57:17.326046 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1a506d-1b41-4384-b359-6526150a77d5 admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:57:17.328285 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-8d1a506d-1b41-4384-b359-6526150a77d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:57:17.385277 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1a506d-1b41-4384-b359-6526150a77d5 admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:57:17.389445 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068d3da1-09df-4e97-b74b-c56f24acdf2c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7922d0a1a7c4e808d0b2a80dca0da40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:57:17.390054 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068d3da1-09df-4e97-b74b-c56f24acdf2c service neutron] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:57:17.390336 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068d3da1-09df-4e97-b74b-c56f24acdf2c service neutron] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:57:17.390669 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068d3da1-09df-4e97-b74b-c56f24acdf2c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:57:17.399487 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068d3da1-09df-4e97-b74b-c56f24acdf2c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:57:17.401045 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068d3da1-09df-4e97-b74b-c56f24acdf2c service neutron] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:57:17.411076 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1378/2731] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:57:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 10:57:17.689077 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e75f297-9ea6-44c8-bf1e-0cd4c7d39eab None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:57:17.690623 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-8e75f297-9ea6-44c8-bf1e-0cd4c7d39eab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:57:17.726953 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e75f297-9ea6-44c8-bf1e-0cd4c7d39eab None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:57:17.732416 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3a305b-bda8-4333-a00d-df6597b7917e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:57:17.733226 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3a305b-bda8-4333-a00d-df6597b7917e admin admin] REQUEST_METHOD: `DELETE` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:57:17.733628 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3a305b-bda8-4333-a00d-df6597b7917e admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:57:17.733977 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3a305b-bda8-4333-a00d-df6597b7917e admin admin] PATH_INFO: `/v3/projects/3074671803cb4acdbab557c073a92f57` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:57:17.743240 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3a305b-bda8-4333-a00d-df6597b7917e admin admin] RBAC: Authorizing `identity:delete_project(project_id=3074671803cb4acdbab557c073a92f57)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:57:17.746007 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3a305b-bda8-4333-a00d-df6597b7917e admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:57:17.797892 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-4f3a305b-bda8-4333-a00d-df6597b7917e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3074671803cb4acdbab557c073a92f57', 'request_id': 'req-4f3a305b-bda8-4333-a00d-df6597b7917e'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:57:17.804453 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-4f3a305b-bda8-4333-a00d-df6597b7917e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3074671803cb4acdbab557c073a92f57', 'request_id': 'req-4f3a305b-bda8-4333-a00d-df6597b7917e'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:57:17.812275 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-4f3a305b-bda8-4333-a00d-df6597b7917e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3074671803cb4acdbab557c073a92f57', 'request_id': 'req-4f3a305b-bda8-4333-a00d-df6597b7917e'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:57:17.822655 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-4f3a305b-bda8-4333-a00d-df6597b7917e admin admin] The token cache is being invalidate because project 3074671803cb4acdbab557c073a92f57 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86828) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 10:57:17.822952 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-4f3a305b-bda8-4333-a00d-df6597b7917e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:57:17.823586 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-4f3a305b-bda8-4333-a00d-df6597b7917e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:57:17.825817 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1354/2732] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Jul 27 10:57:17 2022] DELETE /identity/v3/projects/3074671803cb4acdbab557c073a92f57 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:57:27.406929 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=33,keystone:DELETE=2 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:57:27.819609 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=74,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:57:32.464424 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f68027-e178-4e70-97b5-35238fee6c58 None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:57:32.464909 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f68027-e178-4e70-97b5-35238fee6c58 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:57:32.465025 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f68027-e178-4e70-97b5-35238fee6c58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:57:32.484745 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-c6f68027-e178-4e70-97b5-35238fee6c58 None None] MFA Rules not processed for user `afd04d5fdc7e44c89edddf97be489db9`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:57:32.486148 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-c6f68027-e178-4e70-97b5-35238fee6c58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:57:32.540155 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1379/2733] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:57:32 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:57:42.533610 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=25 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:58:19.601839 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3a305b-bda8-4333-a00d-df6597b7917e admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:58:19.603304 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-4f3a305b-bda8-4333-a00d-df6597b7917e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:58:19.663546 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3a305b-bda8-4333-a00d-df6597b7917e admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:58:19.667136 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40de986-4d5b-4ad6-8780-64c623dbbb16 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '864d35f268cf45148756109759bb2743', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:58:19.667755 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40de986-4d5b-4ad6-8780-64c623dbbb16 service cinder] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:58:19.668040 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40de986-4d5b-4ad6-8780-64c623dbbb16 service cinder] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:58:19.668318 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40de986-4d5b-4ad6-8780-64c623dbbb16 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:58:19.670060 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-e40de986-4d5b-4ad6-8780-64c623dbbb16 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:58:19.752806 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40de986-4d5b-4ad6-8780-64c623dbbb16 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:58:19.754167 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40de986-4d5b-4ad6-8780-64c623dbbb16 service cinder] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:58:19.774434 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1355/2734] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:58:19 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 10:58:26.591696 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f68027-e178-4e70-97b5-35238fee6c58 None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:58:26.601179 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f68027-e178-4e70-97b5-35238fee6c58 None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:58:26.604652 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdcaf63-c083-4369-8154-119a1615dfee service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '864d35f268cf45148756109759bb2743', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:58:26.605252 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdcaf63-c083-4369-8154-119a1615dfee service cinder] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:58:26.605528 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdcaf63-c083-4369-8154-119a1615dfee service cinder] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:58:26.605798 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdcaf63-c083-4369-8154-119a1615dfee service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:58:26.607517 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-fcdcaf63-c083-4369-8154-119a1615dfee service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:58:26.621486 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdcaf63-c083-4369-8154-119a1615dfee service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:58:26.622945 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdcaf63-c083-4369-8154-119a1615dfee service cinder] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:58:26.634621 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1380/2735] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:58:26 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 10:58:29.771867 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=63 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:58:36.629711 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=6 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:58:38.685839 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40de986-4d5b-4ad6-8780-64c623dbbb16 service cinder] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:58:38.688013 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-e40de986-4d5b-4ad6-8780-64c623dbbb16 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:58:38.758051 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40de986-4d5b-4ad6-8780-64c623dbbb16 service cinder] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:58:38.762494 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc624f2-fc4a-4be1-9f9a-c0388232baab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:58:38.763165 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc624f2-fc4a-4be1-9f9a-c0388232baab admin admin] REQUEST_METHOD: `DELETE` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:58:38.763448 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc624f2-fc4a-4be1-9f9a-c0388232baab admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:58:38.763725 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc624f2-fc4a-4be1-9f9a-c0388232baab admin admin] PATH_INFO: `/v3/users/afd04d5fdc7e44c89edddf97be489db9` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:58:38.765039 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc624f2-fc4a-4be1-9f9a-c0388232baab admin admin] RBAC: Authorizing `identity:delete_user(user_id=afd04d5fdc7e44c89edddf97be489db9)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:58:38.767046 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc624f2-fc4a-4be1-9f9a-c0388232baab admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:58:38.813125 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-cdc624f2-fc4a-4be1-9f9a-c0388232baab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'afd04d5fdc7e44c89edddf97be489db9'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:58:38.820188 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-cdc624f2-fc4a-4be1-9f9a-c0388232baab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'afd04d5fdc7e44c89edddf97be489db9'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:58:38.827414 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-cdc624f2-fc4a-4be1-9f9a-c0388232baab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'afd04d5fdc7e44c89edddf97be489db9'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:58:38.828085 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-cdc624f2-fc4a-4be1-9f9a-c0388232baab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'afd04d5fdc7e44c89edddf97be489db9'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:58:38.828735 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-cdc624f2-fc4a-4be1-9f9a-c0388232baab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'afd04d5fdc7e44c89edddf97be489db9'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:58:38.846127 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1356/2736] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Jul 27 10:58:38 2022] DELETE /identity/v3/users/afd04d5fdc7e44c89edddf97be489db9 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 10:58:39.027472 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdcaf63-c083-4369-8154-119a1615dfee service cinder] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:58:39.039280 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdcaf63-c083-4369-8154-119a1615dfee service cinder] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:58:39.096628 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e31aeba-e5fe-47fa-9663-e4e5bb2705fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:58:39.097194 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e31aeba-e5fe-47fa-9663-e4e5bb2705fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:58:39.097480 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e31aeba-e5fe-47fa-9663-e4e5bb2705fd admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:58:39.097758 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e31aeba-e5fe-47fa-9663-e4e5bb2705fd admin admin] PATH_INFO: `/v3/projects/069d5edd9d9346bdb10c80922a763605` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:58:39.098945 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e31aeba-e5fe-47fa-9663-e4e5bb2705fd admin admin] RBAC: Authorizing `identity:delete_project(project_id=069d5edd9d9346bdb10c80922a763605)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:58:39.100850 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e31aeba-e5fe-47fa-9663-e4e5bb2705fd admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:58:39.146265 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-0e31aeba-e5fe-47fa-9663-e4e5bb2705fd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '069d5edd9d9346bdb10c80922a763605', 'request_id': 'req-0e31aeba-e5fe-47fa-9663-e4e5bb2705fd'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:58:39.152332 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-0e31aeba-e5fe-47fa-9663-e4e5bb2705fd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '069d5edd9d9346bdb10c80922a763605', 'request_id': 'req-0e31aeba-e5fe-47fa-9663-e4e5bb2705fd'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:58:39.159567 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-0e31aeba-e5fe-47fa-9663-e4e5bb2705fd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '069d5edd9d9346bdb10c80922a763605', 'request_id': 'req-0e31aeba-e5fe-47fa-9663-e4e5bb2705fd'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:58:39.170042 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-0e31aeba-e5fe-47fa-9663-e4e5bb2705fd admin admin] The token cache is being invalidate because project 069d5edd9d9346bdb10c80922a763605 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86827) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 10:58:39.170368 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-0e31aeba-e5fe-47fa-9663-e4e5bb2705fd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:58:39.171049 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-0e31aeba-e5fe-47fa-9663-e4e5bb2705fd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:58:39.172546 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1381/2737] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Jul 27 10:58:39 2022] DELETE /identity/v3/projects/069d5edd9d9346bdb10c80922a763605 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:58:39.177433 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749be3d2-07a1-4389-b713-cf4d4ba3fd4d None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:58:39.177726 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749be3d2-07a1-4389-b713-cf4d4ba3fd4d None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:58:39.178004 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749be3d2-07a1-4389-b713-cf4d4ba3fd4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:58:39.198270 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-749be3d2-07a1-4389-b713-cf4d4ba3fd4d None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:58:39.199488 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-749be3d2-07a1-4389-b713-cf4d4ba3fd4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:58:39.257761 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1357/2738] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:58:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:58:39.265596 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1302313f-9718-4923-bb2c-ebc1e582cd62 None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:58:39.265914 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1302313f-9718-4923-bb2c-ebc1e582cd62 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:58:39.266197 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1302313f-9718-4923-bb2c-ebc1e582cd62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:58:39.285829 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-1302313f-9718-4923-bb2c-ebc1e582cd62 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:58:39.287098 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-1302313f-9718-4923-bb2c-ebc1e582cd62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:58:39.294978 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1382/2739] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:58:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:58:39.299518 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749be3d2-07a1-4389-b713-cf4d4ba3fd4d None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:58:39.309102 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749be3d2-07a1-4389-b713-cf4d4ba3fd4d None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:58:39.314372 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3796f604-464b-488c-987f-f54bc403a5c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:58:39.315007 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3796f604-464b-488c-987f-f54bc403a5c1 admin admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:58:39.315289 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3796f604-464b-488c-987f-f54bc403a5c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:58:39.315566 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3796f604-464b-488c-987f-f54bc403a5c1 admin admin] PATH_INFO: `/v3/domains` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:58:39.316257 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3796f604-464b-488c-987f-f54bc403a5c1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86828) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:58:39.316559 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3796f604-464b-488c-987f-f54bc403a5c1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:58:39.318135 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3796f604-464b-488c-987f-f54bc403a5c1 admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:58:39.329378 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1358/2740] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Jul 27 10:58:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:58:39.333521 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1302313f-9718-4923-bb2c-ebc1e582cd62 None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:58:39.343639 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1302313f-9718-4923-bb2c-ebc1e582cd62 None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:58:39.348819 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895164b1-1b3d-4865-ae55-14f08e78a40b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:58:39.349369 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895164b1-1b3d-4865-ae55-14f08e78a40b admin admin] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:58:39.349649 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895164b1-1b3d-4865-ae55-14f08e78a40b admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:58:39.349919 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895164b1-1b3d-4865-ae55-14f08e78a40b admin admin] PATH_INFO: `/v3/projects` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:58:39.350574 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895164b1-1b3d-4865-ae55-14f08e78a40b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:58:39.352471 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895164b1-1b3d-4865-ae55-14f08e78a40b admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:58:39.389016 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1383/2741] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Jul 27 10:58:39 2022] POST /identity/v3/projects => generated 377 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 10:58:39.393247 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3796f604-464b-488c-987f-f54bc403a5c1 admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:58:39.403230 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3796f604-464b-488c-987f-f54bc403a5c1 admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:58:39.406810 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:58:39.461697 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11474dc2-fefe-4fe4-aa41-361d31b4eb46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:58:39.462316 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11474dc2-fefe-4fe4-aa41-361d31b4eb46 admin admin] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:58:39.462624 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11474dc2-fefe-4fe4-aa41-361d31b4eb46 admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:58:39.462912 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11474dc2-fefe-4fe4-aa41-361d31b4eb46 admin admin] PATH_INFO: `/v3/users` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:58:39.463588 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11474dc2-fefe-4fe4-aa41-361d31b4eb46 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:58:39.465501 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11474dc2-fefe-4fe4-aa41-361d31b4eb46 admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:58:39.486646 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1359/2742] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Jul 27 10:58:39 2022] POST /identity/v3/users => generated 289 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 10:58:39.490854 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895164b1-1b3d-4865-ae55-14f08e78a40b admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:58:39.501842 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895164b1-1b3d-4865-ae55-14f08e78a40b admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:58:39.506455 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbee7be-3beb-45b5-8da4-fced454a9440 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:58:39.507110 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbee7be-3beb-45b5-8da4-fced454a9440 admin admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:58:39.507391 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbee7be-3beb-45b5-8da4-fced454a9440 admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:58:39.507667 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbee7be-3beb-45b5-8da4-fced454a9440 admin admin] PATH_INFO: `/v3/roles` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:58:39.508361 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dbee7be-3beb-45b5-8da4-fced454a9440 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:58:39.509932 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dbee7be-3beb-45b5-8da4-fced454a9440 admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:58:39.520059 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1384/2743] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 10:58:39 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 10:58:39.524100 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11474dc2-fefe-4fe4-aa41-361d31b4eb46 admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:58:39.534007 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11474dc2-fefe-4fe4-aa41-361d31b4eb46 admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:58:39.538686 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0c3e73-576b-492c-a585-9f746a1f94ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:58:39.539296 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0c3e73-576b-492c-a585-9f746a1f94ae admin admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:58:39.539582 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0c3e73-576b-492c-a585-9f746a1f94ae admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:58:39.539858 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0c3e73-576b-492c-a585-9f746a1f94ae admin admin] PATH_INFO: `/v3/roles` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:58:39.540578 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0c3e73-576b-492c-a585-9f746a1f94ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:58:39.542171 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0c3e73-576b-492c-a585-9f746a1f94ae admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:58:39.553155 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1360/2744] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 10:58:39 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 10:58:39.557217 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbee7be-3beb-45b5-8da4-fced454a9440 admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:58:39.567758 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbee7be-3beb-45b5-8da4-fced454a9440 admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:58:39.572373 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4a550a-b9c4-47e8-90e0-c7a175f89b12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:58:39.573026 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4a550a-b9c4-47e8-90e0-c7a175f89b12 admin admin] REQUEST_METHOD: `PUT` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:58:39.573309 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4a550a-b9c4-47e8-90e0-c7a175f89b12 admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:58:39.573585 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4a550a-b9c4-47e8-90e0-c7a175f89b12 admin admin] PATH_INFO: `/v3/projects/2f55bd5085764f299f739ec4898443a8/users/6b0478ca8b3b4588bcc5bf77b97009bb/roles/4c9fc0b3baed441da2126be41cc6b04a` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:58:39.587883 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4a550a-b9c4-47e8-90e0-c7a175f89b12 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2f55bd5085764f299f739ec4898443a8, user_id=6b0478ca8b3b4588bcc5bf77b97009bb, role_id=4c9fc0b3baed441da2126be41cc6b04a)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:58:39.590499 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4a550a-b9c4-47e8-90e0-c7a175f89b12 admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:58:39.603771 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1385/2745] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 10:58:39 2022] PUT /identity/v3/projects/2f55bd5085764f299f739ec4898443a8/users/6b0478ca8b3b4588bcc5bf77b97009bb/roles/4c9fc0b3baed441da2126be41cc6b04a => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 10:58:39.615814 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3434eba4-14e4-4ecb-b5b1-4631601adceb None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:58:39.616112 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3434eba4-14e4-4ecb-b5b1-4631601adceb None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:58:39.616396 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3434eba4-14e4-4ecb-b5b1-4631601adceb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:58:39.643864 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-3434eba4-14e4-4ecb-b5b1-4631601adceb None None] MFA Rules not processed for user `6b0478ca8b3b4588bcc5bf77b97009bb`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:58:39.645108 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-3434eba4-14e4-4ecb-b5b1-4631601adceb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:58:39.709599 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1361/2746] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:58:39 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:58:39.714228 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4a550a-b9c4-47e8-90e0-c7a175f89b12 admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:58:39.735168 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:58:39.777678 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4a550a-b9c4-47e8-90e0-c7a175f89b12 admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:58:39.782955 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da67226-4a99-4d7c-af15-c8d13b0d1807 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:58:39.783570 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da67226-4a99-4d7c-af15-c8d13b0d1807 admin admin] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:58:39.783856 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da67226-4a99-4d7c-af15-c8d13b0d1807 admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:58:39.784134 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da67226-4a99-4d7c-af15-c8d13b0d1807 admin admin] PATH_INFO: `/v3/projects` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:58:39.784805 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da67226-4a99-4d7c-af15-c8d13b0d1807 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:58:39.786732 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da67226-4a99-4d7c-af15-c8d13b0d1807 admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:58:39.813286 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1386/2747] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Jul 27 10:58:39 2022] POST /identity/v3/projects => generated 375 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 10:58:39.817522 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3434eba4-14e4-4ecb-b5b1-4631601adceb None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:58:39.829180 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3434eba4-14e4-4ecb-b5b1-4631601adceb None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:58:39.884943 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0945fe71-ef6a-4cba-8b2d-e250e198e280 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:58:39.885565 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0945fe71-ef6a-4cba-8b2d-e250e198e280 admin admin] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:58:39.885852 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0945fe71-ef6a-4cba-8b2d-e250e198e280 admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:58:39.886134 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0945fe71-ef6a-4cba-8b2d-e250e198e280 admin admin] PATH_INFO: `/v3/users` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:58:39.886825 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0945fe71-ef6a-4cba-8b2d-e250e198e280 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:58:39.888803 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0945fe71-ef6a-4cba-8b2d-e250e198e280 admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:58:39.911238 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1362/2748] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Jul 27 10:58:39 2022] POST /identity/v3/users => generated 294 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 10:58:39.915594 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da67226-4a99-4d7c-af15-c8d13b0d1807 admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:58:39.928210 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da67226-4a99-4d7c-af15-c8d13b0d1807 admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:58:39.934509 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ecc548-20bf-4f3d-b293-7143d55e93e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:58:39.935179 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ecc548-20bf-4f3d-b293-7143d55e93e9 admin admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:58:39.935466 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ecc548-20bf-4f3d-b293-7143d55e93e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:58:39.935769 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ecc548-20bf-4f3d-b293-7143d55e93e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:58:39.939977 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ecc548-20bf-4f3d-b293-7143d55e93e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:58:39.939977 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ecc548-20bf-4f3d-b293-7143d55e93e9 admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:58:39.950879 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1387/2749] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 10:58:39 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 10:58:39.955554 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0945fe71-ef6a-4cba-8b2d-e250e198e280 admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:58:39.962930 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:58:39.966723 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0945fe71-ef6a-4cba-8b2d-e250e198e280 admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:58:39.972745 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513581fa-318c-4236-a876-59558219a695 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:58:39.973444 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513581fa-318c-4236-a876-59558219a695 admin admin] REQUEST_METHOD: `PUT` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:58:39.973742 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513581fa-318c-4236-a876-59558219a695 admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:58:39.974013 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513581fa-318c-4236-a876-59558219a695 admin admin] PATH_INFO: `/v3/projects/3e0aed3427bd4fb8880dd7b4908dc144/users/6cdddca080fd49da83bd10f1cdd07283/roles/21156560eb6b436bb567a40b4976f3f4` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:58:39.989915 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513581fa-318c-4236-a876-59558219a695 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e0aed3427bd4fb8880dd7b4908dc144, user_id=6cdddca080fd49da83bd10f1cdd07283, role_id=21156560eb6b436bb567a40b4976f3f4)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:58:39.992593 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513581fa-318c-4236-a876-59558219a695 admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:58:40.006744 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1363/2750] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 10:58:39 2022] PUT /identity/v3/projects/3e0aed3427bd4fb8880dd7b4908dc144/users/6cdddca080fd49da83bd10f1cdd07283/roles/21156560eb6b436bb567a40b4976f3f4 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:58:40.010797 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ecc548-20bf-4f3d-b293-7143d55e93e9 admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:58:40.021347 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ecc548-20bf-4f3d-b293-7143d55e93e9 admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:58:40.077622 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95725d29-6dc1-434d-bb3e-0e163c42bd2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:58:40.078263 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95725d29-6dc1-434d-bb3e-0e163c42bd2a admin admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:58:40.078512 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95725d29-6dc1-434d-bb3e-0e163c42bd2a admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:58:40.078854 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95725d29-6dc1-434d-bb3e-0e163c42bd2a admin admin] PATH_INFO: `/v3/roles` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:58:40.079551 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95725d29-6dc1-434d-bb3e-0e163c42bd2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:58:40.081138 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95725d29-6dc1-434d-bb3e-0e163c42bd2a admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:58:40.090745 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1388/2751] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 10:58:40 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 10:58:40.094955 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513581fa-318c-4236-a876-59558219a695 admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:58:40.105370 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513581fa-318c-4236-a876-59558219a695 admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:58:40.110566 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d016ee-0d9b-4481-8678-78674ffa6d09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:58:40.111251 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d016ee-0d9b-4481-8678-78674ffa6d09 admin admin] REQUEST_METHOD: `PUT` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:58:40.111533 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d016ee-0d9b-4481-8678-78674ffa6d09 admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:58:40.111831 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d016ee-0d9b-4481-8678-78674ffa6d09 admin admin] PATH_INFO: `/v3/projects/3e0aed3427bd4fb8880dd7b4908dc144/users/6cdddca080fd49da83bd10f1cdd07283/roles/21156560eb6b436bb567a40b4976f3f4` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:58:40.113813 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d016ee-0d9b-4481-8678-78674ffa6d09 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e0aed3427bd4fb8880dd7b4908dc144, user_id=6cdddca080fd49da83bd10f1cdd07283, role_id=21156560eb6b436bb567a40b4976f3f4)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:58:40.116481 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d016ee-0d9b-4481-8678-78674ffa6d09 admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:58:40.127539 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1364/2752] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 10:58:40 2022] PUT /identity/v3/projects/3e0aed3427bd4fb8880dd7b4908dc144/users/6cdddca080fd49da83bd10f1cdd07283/roles/21156560eb6b436bb567a40b4976f3f4 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:58:40.137641 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551b59ff-0df5-4df6-8ce0-4124da9a4054 None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:58:40.137937 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551b59ff-0df5-4df6-8ce0-4124da9a4054 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:58:40.138223 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551b59ff-0df5-4df6-8ce0-4124da9a4054 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:58:40.165318 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-551b59ff-0df5-4df6-8ce0-4124da9a4054 None None] MFA Rules not processed for user `6cdddca080fd49da83bd10f1cdd07283`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:58:40.166735 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-551b59ff-0df5-4df6-8ce0-4124da9a4054 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:58:40.236115 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1389/2753] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:58:40 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:58:40.247210 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d016ee-0d9b-4481-8678-78674ffa6d09 admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:58:40.249691 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-62d016ee-0d9b-4481-8678-78674ffa6d09 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:58:40.327973 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d016ee-0d9b-4481-8678-78674ffa6d09 admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:58:40.331685 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c6a115-72ab-4413-8117-8fa089e72acc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '864d35f268cf45148756109759bb2743', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:58:40.332314 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c6a115-72ab-4413-8117-8fa089e72acc service cinder] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:58:40.332597 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c6a115-72ab-4413-8117-8fa089e72acc service cinder] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:58:40.332872 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c6a115-72ab-4413-8117-8fa089e72acc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:58:40.388691 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c6a115-72ab-4413-8117-8fa089e72acc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:58:40.390078 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c6a115-72ab-4413-8117-8fa089e72acc service cinder] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:58:40.398817 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1365/2754] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:58:40 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 10:58:45.015131 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551b59ff-0df5-4df6-8ce0-4124da9a4054 None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:58:45.024940 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551b59ff-0df5-4df6-8ce0-4124da9a4054 None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:58:45.028528 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfeaa18-18e0-472f-9fce-7ae3e07b8e74 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '864d35f268cf45148756109759bb2743', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:58:45.029142 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfeaa18-18e0-472f-9fce-7ae3e07b8e74 service cinder] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:58:45.029426 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfeaa18-18e0-472f-9fce-7ae3e07b8e74 service cinder] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:58:45.029701 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfeaa18-18e0-472f-9fce-7ae3e07b8e74 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:58:45.039323 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfeaa18-18e0-472f-9fce-7ae3e07b8e74 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:58:45.040748 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfeaa18-18e0-472f-9fce-7ae3e07b8e74 service cinder] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:58:45.045735 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:58:45.051899 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1390/2755] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:58:45 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 10:58:50.394798 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=61,keystone:INSERT=2 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:59:05.750394 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c6a115-72ab-4413-8117-8fa089e72acc service cinder] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:59:05.824812 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c6a115-72ab-4413-8117-8fa089e72acc service cinder] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:59:05.829314 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7aeafc-8e0d-4a35-88ff-1f40b7846701 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:59:05.829950 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7aeafc-8e0d-4a35-88ff-1f40b7846701 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:59:05.830233 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7aeafc-8e0d-4a35-88ff-1f40b7846701 admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:59:05.830510 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7aeafc-8e0d-4a35-88ff-1f40b7846701 admin admin] PATH_INFO: `/v3/users/6b0478ca8b3b4588bcc5bf77b97009bb` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:59:05.831700 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7aeafc-8e0d-4a35-88ff-1f40b7846701 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6b0478ca8b3b4588bcc5bf77b97009bb)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:59:05.834172 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7aeafc-8e0d-4a35-88ff-1f40b7846701 admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:59:05.883633 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-db7aeafc-8e0d-4a35-88ff-1f40b7846701 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6b0478ca8b3b4588bcc5bf77b97009bb'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:59:05.890678 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-db7aeafc-8e0d-4a35-88ff-1f40b7846701 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6b0478ca8b3b4588bcc5bf77b97009bb'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:59:05.898720 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-db7aeafc-8e0d-4a35-88ff-1f40b7846701 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6b0478ca8b3b4588bcc5bf77b97009bb'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:59:05.899327 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-db7aeafc-8e0d-4a35-88ff-1f40b7846701 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6b0478ca8b3b4588bcc5bf77b97009bb'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:59:05.899938 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-db7aeafc-8e0d-4a35-88ff-1f40b7846701 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6b0478ca8b3b4588bcc5bf77b97009bb'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:59:05.917664 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1366/2756] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Jul 27 10:59:05 2022] DELETE /identity/v3/users/6b0478ca8b3b4588bcc5bf77b97009bb => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:59:05.922110 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3aa60bd-baa1-4690-aa86-6815da5e8906 None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:59:05.922583 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3aa60bd-baa1-4690-aa86-6815da5e8906 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:59:05.922727 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3aa60bd-baa1-4690-aa86-6815da5e8906 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:59:05.944496 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-b3aa60bd-baa1-4690-aa86-6815da5e8906 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:59:05.945587 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-b3aa60bd-baa1-4690-aa86-6815da5e8906 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:59:06.003338 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1391/2757] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:59:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:59:06.013478 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7aeafc-8e0d-4a35-88ff-1f40b7846701 admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:59:06.014914 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-db7aeafc-8e0d-4a35-88ff-1f40b7846701 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:59:06.072115 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7aeafc-8e0d-4a35-88ff-1f40b7846701 admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:59:06.075820 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee4ca1d-87d4-4290-a15f-23f4ba2e27d1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7922d0a1a7c4e808d0b2a80dca0da40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:59:06.076472 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee4ca1d-87d4-4290-a15f-23f4ba2e27d1 service neutron] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:59:06.076759 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee4ca1d-87d4-4290-a15f-23f4ba2e27d1 service neutron] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:59:06.077035 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee4ca1d-87d4-4290-a15f-23f4ba2e27d1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:59:06.086872 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee4ca1d-87d4-4290-a15f-23f4ba2e27d1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:59:06.088311 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee4ca1d-87d4-4290-a15f-23f4ba2e27d1 service neutron] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:59:06.097791 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1367/2758] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:59:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 10:59:06.363117 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3aa60bd-baa1-4690-aa86-6815da5e8906 None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:59:06.365173 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-b3aa60bd-baa1-4690-aa86-6815da5e8906 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:59:06.380855 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3aa60bd-baa1-4690-aa86-6815da5e8906 None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:59:06.386539 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422b8008-da50-4e06-8c12-2c8ce1fa567b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:59:06.387331 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422b8008-da50-4e06-8c12-2c8ce1fa567b admin admin] REQUEST_METHOD: `DELETE` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:59:06.387492 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422b8008-da50-4e06-8c12-2c8ce1fa567b admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:59:06.387754 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422b8008-da50-4e06-8c12-2c8ce1fa567b admin admin] PATH_INFO: `/v3/projects/2f55bd5085764f299f739ec4898443a8` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:59:06.388919 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422b8008-da50-4e06-8c12-2c8ce1fa567b admin admin] RBAC: Authorizing `identity:delete_project(project_id=2f55bd5085764f299f739ec4898443a8)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:59:06.390969 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422b8008-da50-4e06-8c12-2c8ce1fa567b admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:59:06.439435 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-422b8008-da50-4e06-8c12-2c8ce1fa567b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2f55bd5085764f299f739ec4898443a8', 'request_id': 'req-422b8008-da50-4e06-8c12-2c8ce1fa567b'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:59:06.446207 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-422b8008-da50-4e06-8c12-2c8ce1fa567b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2f55bd5085764f299f739ec4898443a8', 'request_id': 'req-422b8008-da50-4e06-8c12-2c8ce1fa567b'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:59:06.453113 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-422b8008-da50-4e06-8c12-2c8ce1fa567b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2f55bd5085764f299f739ec4898443a8', 'request_id': 'req-422b8008-da50-4e06-8c12-2c8ce1fa567b'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:59:06.463689 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-422b8008-da50-4e06-8c12-2c8ce1fa567b admin admin] The token cache is being invalidate because project 2f55bd5085764f299f739ec4898443a8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86827) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 10:59:06.463977 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-422b8008-da50-4e06-8c12-2c8ce1fa567b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:59:06.464591 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-422b8008-da50-4e06-8c12-2c8ce1fa567b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:59:06.466196 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1392/2759] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Jul 27 10:59:06 2022] DELETE /identity/v3/projects/2f55bd5085764f299f739ec4898443a8 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:59:06.470141 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee4ca1d-87d4-4290-a15f-23f4ba2e27d1 service neutron] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:59:06.481617 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee4ca1d-87d4-4290-a15f-23f4ba2e27d1 service neutron] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:59:06.500243 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:59:06.539286 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55794528-9662-4480-be31-a7388212f73d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:59:06.539848 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55794528-9662-4480-be31-a7388212f73d admin admin] REQUEST_METHOD: `DELETE` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:59:06.540130 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55794528-9662-4480-be31-a7388212f73d admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:59:06.540402 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55794528-9662-4480-be31-a7388212f73d admin admin] PATH_INFO: `/v3/users/6cdddca080fd49da83bd10f1cdd07283` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:59:06.541550 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55794528-9662-4480-be31-a7388212f73d admin admin] RBAC: Authorizing `identity:delete_user(user_id=6cdddca080fd49da83bd10f1cdd07283)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:59:06.543503 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55794528-9662-4480-be31-a7388212f73d admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:59:06.590863 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-55794528-9662-4480-be31-a7388212f73d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6cdddca080fd49da83bd10f1cdd07283'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:59:06.597553 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-55794528-9662-4480-be31-a7388212f73d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6cdddca080fd49da83bd10f1cdd07283'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:59:06.604562 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-55794528-9662-4480-be31-a7388212f73d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6cdddca080fd49da83bd10f1cdd07283'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:59:06.605253 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-55794528-9662-4480-be31-a7388212f73d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6cdddca080fd49da83bd10f1cdd07283'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:59:06.605858 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-55794528-9662-4480-be31-a7388212f73d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6cdddca080fd49da83bd10f1cdd07283'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:59:06.623785 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1368/2760] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Jul 27 10:59:06 2022] DELETE /identity/v3/users/6cdddca080fd49da83bd10f1cdd07283 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:59:06.897629 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422b8008-da50-4e06-8c12-2c8ce1fa567b admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:59:06.899222 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-422b8008-da50-4e06-8c12-2c8ce1fa567b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:59:06.916570 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422b8008-da50-4e06-8c12-2c8ce1fa567b admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:59:06.974464 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e343d8b1-f4ba-49d1-aafc-fd5953951654 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:59:06.975156 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e343d8b1-f4ba-49d1-aafc-fd5953951654 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:59:06.975415 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e343d8b1-f4ba-49d1-aafc-fd5953951654 admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:59:06.975694 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e343d8b1-f4ba-49d1-aafc-fd5953951654 admin admin] PATH_INFO: `/v3/projects/3e0aed3427bd4fb8880dd7b4908dc144` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:59:06.976853 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e343d8b1-f4ba-49d1-aafc-fd5953951654 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3e0aed3427bd4fb8880dd7b4908dc144)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:59:06.978831 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e343d8b1-f4ba-49d1-aafc-fd5953951654 admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:59:07.009496 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:59:07.026848 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-e343d8b1-f4ba-49d1-aafc-fd5953951654 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3e0aed3427bd4fb8880dd7b4908dc144', 'request_id': 'req-e343d8b1-f4ba-49d1-aafc-fd5953951654'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:59:07.032769 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-e343d8b1-f4ba-49d1-aafc-fd5953951654 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3e0aed3427bd4fb8880dd7b4908dc144', 'request_id': 'req-e343d8b1-f4ba-49d1-aafc-fd5953951654'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:59:07.039898 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-e343d8b1-f4ba-49d1-aafc-fd5953951654 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3e0aed3427bd4fb8880dd7b4908dc144', 'request_id': 'req-e343d8b1-f4ba-49d1-aafc-fd5953951654'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:59:07.050017 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-e343d8b1-f4ba-49d1-aafc-fd5953951654 admin admin] The token cache is being invalidate because project 3e0aed3427bd4fb8880dd7b4908dc144 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86827) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 10:59:07.050302 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-e343d8b1-f4ba-49d1-aafc-fd5953951654 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:59:07.051435 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-e343d8b1-f4ba-49d1-aafc-fd5953951654 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:59:07.052919 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1393/2761] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Jul 27 10:59:06 2022] DELETE /identity/v3/projects/3e0aed3427bd4fb8880dd7b4908dc144 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:59:07.057726 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0705bcb6-c97a-4090-97fc-f8c4b59a35dc None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:59:07.058209 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0705bcb6-c97a-4090-97fc-f8c4b59a35dc None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:59:07.058309 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0705bcb6-c97a-4090-97fc-f8c4b59a35dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:59:07.078116 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-0705bcb6-c97a-4090-97fc-f8c4b59a35dc None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:59:07.079257 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-0705bcb6-c97a-4090-97fc-f8c4b59a35dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:59:07.137727 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1369/2762] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:59:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:59:07.146839 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf46754f-5caa-48c4-8b4b-ba974c459bee None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:59:07.147080 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf46754f-5caa-48c4-8b4b-ba974c459bee None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:59:07.147357 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf46754f-5caa-48c4-8b4b-ba974c459bee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:59:07.167189 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-cf46754f-5caa-48c4-8b4b-ba974c459bee None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:59:07.168423 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-cf46754f-5caa-48c4-8b4b-ba974c459bee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:59:07.176885 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1394/2763] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:59:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:59:07.181041 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0705bcb6-c97a-4090-97fc-f8c4b59a35dc None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:59:07.190589 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0705bcb6-c97a-4090-97fc-f8c4b59a35dc None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:59:07.195911 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a40f44d-e85c-4306-99b3-172053f7e8c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:59:07.196522 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a40f44d-e85c-4306-99b3-172053f7e8c7 admin admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:59:07.196807 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a40f44d-e85c-4306-99b3-172053f7e8c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:59:07.197084 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a40f44d-e85c-4306-99b3-172053f7e8c7 admin admin] PATH_INFO: `/v3/domains` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:59:07.197777 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a40f44d-e85c-4306-99b3-172053f7e8c7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86828) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:59:07.198082 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a40f44d-e85c-4306-99b3-172053f7e8c7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:59:07.199720 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a40f44d-e85c-4306-99b3-172053f7e8c7 admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:59:07.211085 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1370/2764] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Jul 27 10:59:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:59:07.215205 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf46754f-5caa-48c4-8b4b-ba974c459bee None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:59:07.224999 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf46754f-5caa-48c4-8b4b-ba974c459bee None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:59:07.229679 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337c7415-372e-4aad-b945-731610f2cd0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:59:07.230282 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337c7415-372e-4aad-b945-731610f2cd0e admin admin] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:59:07.230564 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337c7415-372e-4aad-b945-731610f2cd0e admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:59:07.230882 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337c7415-372e-4aad-b945-731610f2cd0e admin admin] PATH_INFO: `/v3/projects` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:59:07.231556 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337c7415-372e-4aad-b945-731610f2cd0e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:59:07.233443 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337c7415-372e-4aad-b945-731610f2cd0e admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:59:07.261468 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1395/2765] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Jul 27 10:59:07 2022] POST /identity/v3/projects => generated 389 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 10:59:07.265548 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a40f44d-e85c-4306-99b3-172053f7e8c7 admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:59:07.275746 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a40f44d-e85c-4306-99b3-172053f7e8c7 admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:59:07.301992 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:59:07.332497 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905facdc-d0f6-4772-9df9-f2e65cdb78cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:59:07.333043 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905facdc-d0f6-4772-9df9-f2e65cdb78cd admin admin] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:59:07.333321 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905facdc-d0f6-4772-9df9-f2e65cdb78cd admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:59:07.333592 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905facdc-d0f6-4772-9df9-f2e65cdb78cd admin admin] PATH_INFO: `/v3/users` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:59:07.334247 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905facdc-d0f6-4772-9df9-f2e65cdb78cd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:59:07.336207 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905facdc-d0f6-4772-9df9-f2e65cdb78cd admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:59:07.356646 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1371/2766] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Jul 27 10:59:07 2022] POST /identity/v3/users => generated 295 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 10:59:07.360814 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337c7415-372e-4aad-b945-731610f2cd0e admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:59:07.371943 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337c7415-372e-4aad-b945-731610f2cd0e admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:59:07.376550 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b128e3-5528-444e-b1ba-defac8a1ba7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:59:07.377159 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b128e3-5528-444e-b1ba-defac8a1ba7a admin admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:59:07.377446 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b128e3-5528-444e-b1ba-defac8a1ba7a admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:59:07.377726 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b128e3-5528-444e-b1ba-defac8a1ba7a admin admin] PATH_INFO: `/v3/roles` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:59:07.378431 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b128e3-5528-444e-b1ba-defac8a1ba7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:59:07.380076 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b128e3-5528-444e-b1ba-defac8a1ba7a admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:59:07.390047 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1396/2767] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 10:59:07 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 10:59:07.393906 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905facdc-d0f6-4772-9df9-f2e65cdb78cd admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:59:07.403705 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905facdc-d0f6-4772-9df9-f2e65cdb78cd admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:59:07.408235 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44509058-f574-4fad-b2f5-1cecbf4637e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:59:07.408833 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44509058-f574-4fad-b2f5-1cecbf4637e5 admin admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:59:07.409112 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44509058-f574-4fad-b2f5-1cecbf4637e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:59:07.409382 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44509058-f574-4fad-b2f5-1cecbf4637e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:59:07.410062 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44509058-f574-4fad-b2f5-1cecbf4637e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:59:07.412152 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44509058-f574-4fad-b2f5-1cecbf4637e5 admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:59:07.422353 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1372/2768] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 10:59:07 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 10:59:07.427034 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b128e3-5528-444e-b1ba-defac8a1ba7a admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:59:07.437143 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b128e3-5528-444e-b1ba-defac8a1ba7a admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:59:07.441890 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fa6005-6549-4ee3-9a2e-a1302314593b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:59:07.442669 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fa6005-6549-4ee3-9a2e-a1302314593b admin admin] REQUEST_METHOD: `PUT` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:59:07.442950 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fa6005-6549-4ee3-9a2e-a1302314593b admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:59:07.443342 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fa6005-6549-4ee3-9a2e-a1302314593b admin admin] PATH_INFO: `/v3/projects/6ee5a81f8dd6460b9d2c7ab2f55cf2df/users/bd0ad50d4b4b4613871a6054e794561f/roles/4c9fc0b3baed441da2126be41cc6b04a` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:59:07.458294 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35fa6005-6549-4ee3-9a2e-a1302314593b admin admin] RBAC: Authorizing `identity:create_grant(project_id=6ee5a81f8dd6460b9d2c7ab2f55cf2df, user_id=bd0ad50d4b4b4613871a6054e794561f, role_id=4c9fc0b3baed441da2126be41cc6b04a)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:59:07.461141 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35fa6005-6549-4ee3-9a2e-a1302314593b admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:59:07.475249 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1397/2769] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 10:59:07 2022] PUT /identity/v3/projects/6ee5a81f8dd6460b9d2c7ab2f55cf2df/users/bd0ad50d4b4b4613871a6054e794561f/roles/4c9fc0b3baed441da2126be41cc6b04a => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:59:07.485699 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2feba4a1-4f7e-4cd8-9c27-ee3b3d6f1c7e None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:59:07.485985 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2feba4a1-4f7e-4cd8-9c27-ee3b3d6f1c7e None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:59:07.486261 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2feba4a1-4f7e-4cd8-9c27-ee3b3d6f1c7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:59:07.514106 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-2feba4a1-4f7e-4cd8-9c27-ee3b3d6f1c7e None None] MFA Rules not processed for user `bd0ad50d4b4b4613871a6054e794561f`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:59:07.515323 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-2feba4a1-4f7e-4cd8-9c27-ee3b3d6f1c7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:59:07.583901 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1373/2770] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:59:07 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:59:07.588359 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fa6005-6549-4ee3-9a2e-a1302314593b admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:59:07.650391 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fa6005-6549-4ee3-9a2e-a1302314593b admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:59:07.655224 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02230c7-f815-474a-97eb-7c6d2e88140d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:59:07.655838 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02230c7-f815-474a-97eb-7c6d2e88140d admin admin] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:59:07.656145 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02230c7-f815-474a-97eb-7c6d2e88140d admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:59:07.656420 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02230c7-f815-474a-97eb-7c6d2e88140d admin admin] PATH_INFO: `/v3/projects` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:59:07.657094 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02230c7-f815-474a-97eb-7c6d2e88140d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:59:07.659037 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02230c7-f815-474a-97eb-7c6d2e88140d admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:59:07.685607 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1398/2771] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Jul 27 10:59:07 2022] POST /identity/v3/projects => generated 389 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 10:59:07.689686 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2feba4a1-4f7e-4cd8-9c27-ee3b3d6f1c7e None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:59:07.700220 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2feba4a1-4f7e-4cd8-9c27-ee3b3d6f1c7e None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:59:07.758085 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5052f958-9adc-43af-93a1-0f5ab76dfb5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:59:07.758762 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5052f958-9adc-43af-93a1-0f5ab76dfb5d admin admin] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:59:07.759050 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5052f958-9adc-43af-93a1-0f5ab76dfb5d admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:59:07.759336 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5052f958-9adc-43af-93a1-0f5ab76dfb5d admin admin] PATH_INFO: `/v3/users` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:59:07.760033 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5052f958-9adc-43af-93a1-0f5ab76dfb5d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:59:07.761947 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5052f958-9adc-43af-93a1-0f5ab76dfb5d admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:59:07.784781 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1374/2772] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Jul 27 10:59:07 2022] POST /identity/v3/users => generated 301 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 10:59:07.789985 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02230c7-f815-474a-97eb-7c6d2e88140d admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:59:07.801018 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02230c7-f815-474a-97eb-7c6d2e88140d admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:59:07.805602 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8949b1-0ac6-49ce-b1ac-e66b0fa48a7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:59:07.806214 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8949b1-0ac6-49ce-b1ac-e66b0fa48a7a admin admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:59:07.806492 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8949b1-0ac6-49ce-b1ac-e66b0fa48a7a admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:59:07.806795 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8949b1-0ac6-49ce-b1ac-e66b0fa48a7a admin admin] PATH_INFO: `/v3/roles` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:59:07.807491 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8949b1-0ac6-49ce-b1ac-e66b0fa48a7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:59:07.809076 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8949b1-0ac6-49ce-b1ac-e66b0fa48a7a admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:59:07.819704 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1399/2773] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 10:59:07 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 10:59:07.823834 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5052f958-9adc-43af-93a1-0f5ab76dfb5d admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:59:07.834568 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5052f958-9adc-43af-93a1-0f5ab76dfb5d admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:59:07.839317 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14967333-c9b7-420c-9f6d-873faa7b368e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:59:07.839989 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14967333-c9b7-420c-9f6d-873faa7b368e admin admin] REQUEST_METHOD: `PUT` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:59:07.840274 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14967333-c9b7-420c-9f6d-873faa7b368e admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:59:07.840554 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14967333-c9b7-420c-9f6d-873faa7b368e admin admin] PATH_INFO: `/v3/projects/c5c5b461e4a04820b01f62745148feca/users/dd65f81ac8f4478bbcea6f28cc96c84b/roles/21156560eb6b436bb567a40b4976f3f4` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:59:07.856552 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14967333-c9b7-420c-9f6d-873faa7b368e admin admin] RBAC: Authorizing `identity:create_grant(project_id=c5c5b461e4a04820b01f62745148feca, user_id=dd65f81ac8f4478bbcea6f28cc96c84b, role_id=21156560eb6b436bb567a40b4976f3f4)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:59:07.859406 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14967333-c9b7-420c-9f6d-873faa7b368e admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:59:07.874303 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1375/2774] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 10:59:07 2022] PUT /identity/v3/projects/c5c5b461e4a04820b01f62745148feca/users/dd65f81ac8f4478bbcea6f28cc96c84b/roles/21156560eb6b436bb567a40b4976f3f4 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:59:07.878251 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8949b1-0ac6-49ce-b1ac-e66b0fa48a7a admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:59:07.888372 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8949b1-0ac6-49ce-b1ac-e66b0fa48a7a admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:59:07.908569 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:59:07.948233 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a935ba2a-0fa8-4e32-a058-6e6d1439688c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:59:07.948857 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a935ba2a-0fa8-4e32-a058-6e6d1439688c admin admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:59:07.949143 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a935ba2a-0fa8-4e32-a058-6e6d1439688c admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:59:07.949449 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a935ba2a-0fa8-4e32-a058-6e6d1439688c admin admin] PATH_INFO: `/v3/roles` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:59:07.950151 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a935ba2a-0fa8-4e32-a058-6e6d1439688c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:59:07.951774 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a935ba2a-0fa8-4e32-a058-6e6d1439688c admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:59:07.962334 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1400/2775] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 10:59:07 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 10:59:07.966517 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14967333-c9b7-420c-9f6d-873faa7b368e admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:59:07.972019 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:59:07.979194 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14967333-c9b7-420c-9f6d-873faa7b368e admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:59:07.983968 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bb3f37-39e1-4443-90c6-159a6c4123da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:59:07.984633 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bb3f37-39e1-4443-90c6-159a6c4123da admin admin] REQUEST_METHOD: `PUT` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:59:07.984966 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bb3f37-39e1-4443-90c6-159a6c4123da admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:59:07.985232 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bb3f37-39e1-4443-90c6-159a6c4123da admin admin] PATH_INFO: `/v3/projects/c5c5b461e4a04820b01f62745148feca/users/dd65f81ac8f4478bbcea6f28cc96c84b/roles/21156560eb6b436bb567a40b4976f3f4` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:59:07.987219 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09bb3f37-39e1-4443-90c6-159a6c4123da admin admin] RBAC: Authorizing `identity:create_grant(project_id=c5c5b461e4a04820b01f62745148feca, user_id=dd65f81ac8f4478bbcea6f28cc96c84b, role_id=21156560eb6b436bb567a40b4976f3f4)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:59:07.989907 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09bb3f37-39e1-4443-90c6-159a6c4123da admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:59:08.000294 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1376/2776] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 10:59:07 2022] PUT /identity/v3/projects/c5c5b461e4a04820b01f62745148feca/users/dd65f81ac8f4478bbcea6f28cc96c84b/roles/21156560eb6b436bb567a40b4976f3f4 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:59:08.011792 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ea4b8f-9046-4763-9dc0-140f7adbe7f1 None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:59:08.012260 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ea4b8f-9046-4763-9dc0-140f7adbe7f1 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:59:08.012369 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ea4b8f-9046-4763-9dc0-140f7adbe7f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:59:08.039861 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-48ea4b8f-9046-4763-9dc0-140f7adbe7f1 None None] MFA Rules not processed for user `dd65f81ac8f4478bbcea6f28cc96c84b`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:59:08.041058 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-48ea4b8f-9046-4763-9dc0-140f7adbe7f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:59:08.108666 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1401/2777] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:59:08 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:59:08.119100 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bb3f37-39e1-4443-90c6-159a6c4123da admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:59:08.121159 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-09bb3f37-39e1-4443-90c6-159a6c4123da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:59:08.178648 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bb3f37-39e1-4443-90c6-159a6c4123da admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:59:08.182275 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf3eeb0-fb57-4607-9e2d-e16595f34159 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '864d35f268cf45148756109759bb2743', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:59:08.182988 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf3eeb0-fb57-4607-9e2d-e16595f34159 service cinder] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:59:08.183276 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf3eeb0-fb57-4607-9e2d-e16595f34159 service cinder] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:59:08.183556 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf3eeb0-fb57-4607-9e2d-e16595f34159 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:59:08.193268 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf3eeb0-fb57-4607-9e2d-e16595f34159 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:59:08.194721 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf3eeb0-fb57-4607-9e2d-e16595f34159 service cinder] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:59:08.203999 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1377/2778] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:59:08 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 10:59:08.335826 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ea4b8f-9046-4763-9dc0-140f7adbe7f1 None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:59:08.396280 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ea4b8f-9046-4763-9dc0-140f7adbe7f1 None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:59:08.400672 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a163fa63-e9b9-48be-a302-d27896f0f421 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:59:08.401301 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a163fa63-e9b9-48be-a302-d27896f0f421 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:59:08.401585 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a163fa63-e9b9-48be-a302-d27896f0f421 admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:59:08.401861 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a163fa63-e9b9-48be-a302-d27896f0f421 admin admin] PATH_INFO: `/v3/users/bd0ad50d4b4b4613871a6054e794561f` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:59:08.403042 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a163fa63-e9b9-48be-a302-d27896f0f421 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bd0ad50d4b4b4613871a6054e794561f)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:59:08.404946 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a163fa63-e9b9-48be-a302-d27896f0f421 admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:59:08.450305 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-a163fa63-e9b9-48be-a302-d27896f0f421 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bd0ad50d4b4b4613871a6054e794561f'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:59:08.451544 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:59:08.459282 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-a163fa63-e9b9-48be-a302-d27896f0f421 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bd0ad50d4b4b4613871a6054e794561f'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:59:08.466496 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-a163fa63-e9b9-48be-a302-d27896f0f421 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bd0ad50d4b4b4613871a6054e794561f'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:59:08.467150 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-a163fa63-e9b9-48be-a302-d27896f0f421 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bd0ad50d4b4b4613871a6054e794561f'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:59:08.467779 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-a163fa63-e9b9-48be-a302-d27896f0f421 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bd0ad50d4b4b4613871a6054e794561f'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:59:08.485492 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1402/2779] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Jul 27 10:59:08 2022] DELETE /identity/v3/users/bd0ad50d4b4b4613871a6054e794561f => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:59:08.490991 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e786e4-f32f-408e-bb62-2a4aae5f1950 None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:59:08.491410 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e786e4-f32f-408e-bb62-2a4aae5f1950 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:59:08.491688 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e786e4-f32f-408e-bb62-2a4aae5f1950 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:59:08.513227 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-02e786e4-f32f-408e-bb62-2a4aae5f1950 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:59:08.514416 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-02e786e4-f32f-408e-bb62-2a4aae5f1950 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:59:08.572579 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1378/2780] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:59:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:59:08.582683 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a163fa63-e9b9-48be-a302-d27896f0f421 admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:59:08.584076 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-a163fa63-e9b9-48be-a302-d27896f0f421 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:59:08.640702 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a163fa63-e9b9-48be-a302-d27896f0f421 admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:59:08.644308 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17c2cf5-8ee6-412f-8bd0-259169827a22 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7922d0a1a7c4e808d0b2a80dca0da40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:59:08.645410 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17c2cf5-8ee6-412f-8bd0-259169827a22 service neutron] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:59:08.645685 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17c2cf5-8ee6-412f-8bd0-259169827a22 service neutron] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:59:08.645955 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17c2cf5-8ee6-412f-8bd0-259169827a22 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:59:08.654835 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17c2cf5-8ee6-412f-8bd0-259169827a22 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:59:08.656215 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17c2cf5-8ee6-412f-8bd0-259169827a22 service neutron] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:59:08.666224 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1403/2781] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:59:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 10:59:08.929903 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e786e4-f32f-408e-bb62-2a4aae5f1950 None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:59:08.931346 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-02e786e4-f32f-408e-bb62-2a4aae5f1950 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:59:08.957093 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e786e4-f32f-408e-bb62-2a4aae5f1950 None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:59:08.964280 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537826ec-43cf-4515-b6ba-36f05519b932 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:59:08.964939 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537826ec-43cf-4515-b6ba-36f05519b932 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:59:08.965223 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537826ec-43cf-4515-b6ba-36f05519b932 admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:59:08.965500 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537826ec-43cf-4515-b6ba-36f05519b932 admin admin] PATH_INFO: `/v3/projects/6ee5a81f8dd6460b9d2c7ab2f55cf2df` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:59:08.966710 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537826ec-43cf-4515-b6ba-36f05519b932 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6ee5a81f8dd6460b9d2c7ab2f55cf2df)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:59:08.968617 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537826ec-43cf-4515-b6ba-36f05519b932 admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:59:09.014805 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-537826ec-43cf-4515-b6ba-36f05519b932 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6ee5a81f8dd6460b9d2c7ab2f55cf2df', 'request_id': 'req-537826ec-43cf-4515-b6ba-36f05519b932'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:59:09.021384 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-537826ec-43cf-4515-b6ba-36f05519b932 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6ee5a81f8dd6460b9d2c7ab2f55cf2df', 'request_id': 'req-537826ec-43cf-4515-b6ba-36f05519b932'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:59:09.028697 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-537826ec-43cf-4515-b6ba-36f05519b932 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6ee5a81f8dd6460b9d2c7ab2f55cf2df', 'request_id': 'req-537826ec-43cf-4515-b6ba-36f05519b932'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:59:09.038848 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-537826ec-43cf-4515-b6ba-36f05519b932 admin admin] The token cache is being invalidate because project 6ee5a81f8dd6460b9d2c7ab2f55cf2df was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86828) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 10:59:09.039133 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-537826ec-43cf-4515-b6ba-36f05519b932 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:59:09.039740 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-537826ec-43cf-4515-b6ba-36f05519b932 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:59:09.041251 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1379/2782] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Jul 27 10:59:08 2022] DELETE /identity/v3/projects/6ee5a81f8dd6460b9d2c7ab2f55cf2df => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:59:09.100445 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17c2cf5-8ee6-412f-8bd0-259169827a22 service neutron] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:59:09.110676 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17c2cf5-8ee6-412f-8bd0-259169827a22 service neutron] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:59:09.166886 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f050da-3978-4340-902b-396f36b3e2ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:59:09.167520 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f050da-3978-4340-902b-396f36b3e2ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:59:09.167806 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f050da-3978-4340-902b-396f36b3e2ca admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:59:09.168086 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f050da-3978-4340-902b-396f36b3e2ca admin admin] PATH_INFO: `/v3/users/dd65f81ac8f4478bbcea6f28cc96c84b` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:59:09.169274 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f050da-3978-4340-902b-396f36b3e2ca admin admin] RBAC: Authorizing `identity:delete_user(user_id=dd65f81ac8f4478bbcea6f28cc96c84b)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:59:09.171228 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f050da-3978-4340-902b-396f36b3e2ca admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:59:09.217267 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-64f050da-3978-4340-902b-396f36b3e2ca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dd65f81ac8f4478bbcea6f28cc96c84b'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:59:09.224157 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-64f050da-3978-4340-902b-396f36b3e2ca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dd65f81ac8f4478bbcea6f28cc96c84b'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:59:09.231443 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-64f050da-3978-4340-902b-396f36b3e2ca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dd65f81ac8f4478bbcea6f28cc96c84b'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:59:09.232062 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-64f050da-3978-4340-902b-396f36b3e2ca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dd65f81ac8f4478bbcea6f28cc96c84b'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:59:09.232670 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-64f050da-3978-4340-902b-396f36b3e2ca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dd65f81ac8f4478bbcea6f28cc96c84b'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:59:09.237869 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=86,keystone:INSERT=2,keystone:DELETE=12 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:59:09.253117 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1404/2783] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Jul 27 10:59:09 2022] DELETE /identity/v3/users/dd65f81ac8f4478bbcea6f28cc96c84b => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:59:09.751939 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537826ec-43cf-4515-b6ba-36f05519b932 admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:59:09.753550 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-537826ec-43cf-4515-b6ba-36f05519b932 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:59:09.767194 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537826ec-43cf-4515-b6ba-36f05519b932 admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:59:09.780782 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:59:09.828084 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd0fa91-54ec-4ed8-9cae-5828dd9e970a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:59:09.828721 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd0fa91-54ec-4ed8-9cae-5828dd9e970a admin admin] REQUEST_METHOD: `DELETE` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:59:09.828999 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd0fa91-54ec-4ed8-9cae-5828dd9e970a admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:59:09.829271 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd0fa91-54ec-4ed8-9cae-5828dd9e970a admin admin] PATH_INFO: `/v3/projects/c5c5b461e4a04820b01f62745148feca` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:59:09.830405 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd0fa91-54ec-4ed8-9cae-5828dd9e970a admin admin] RBAC: Authorizing `identity:delete_project(project_id=c5c5b461e4a04820b01f62745148feca)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:59:09.832819 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd0fa91-54ec-4ed8-9cae-5828dd9e970a admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:59:09.878680 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-1bd0fa91-54ec-4ed8-9cae-5828dd9e970a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c5c5b461e4a04820b01f62745148feca', 'request_id': 'req-1bd0fa91-54ec-4ed8-9cae-5828dd9e970a'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:59:09.884614 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-1bd0fa91-54ec-4ed8-9cae-5828dd9e970a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c5c5b461e4a04820b01f62745148feca', 'request_id': 'req-1bd0fa91-54ec-4ed8-9cae-5828dd9e970a'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:59:09.892787 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-1bd0fa91-54ec-4ed8-9cae-5828dd9e970a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c5c5b461e4a04820b01f62745148feca', 'request_id': 'req-1bd0fa91-54ec-4ed8-9cae-5828dd9e970a'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:59:09.903704 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-1bd0fa91-54ec-4ed8-9cae-5828dd9e970a admin admin] The token cache is being invalidate because project c5c5b461e4a04820b01f62745148feca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86828) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 10:59:09.903997 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-1bd0fa91-54ec-4ed8-9cae-5828dd9e970a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:59:09.904612 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-1bd0fa91-54ec-4ed8-9cae-5828dd9e970a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:59:09.906127 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1380/2784] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Jul 27 10:59:09 2022] DELETE /identity/v3/projects/c5c5b461e4a04820b01f62745148feca => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:59:09.911755 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02150802-6678-417d-88f7-4dc406374af7 None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:59:09.912147 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02150802-6678-417d-88f7-4dc406374af7 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:59:09.912561 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02150802-6678-417d-88f7-4dc406374af7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:59:09.933102 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-02150802-6678-417d-88f7-4dc406374af7 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:59:09.934299 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-02150802-6678-417d-88f7-4dc406374af7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:59:09.994197 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1405/2785] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:59:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:59:10.001804 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1435cd3b-0942-437c-9119-480b5ab3fc96 None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:59:10.002097 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1435cd3b-0942-437c-9119-480b5ab3fc96 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:59:10.002377 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1435cd3b-0942-437c-9119-480b5ab3fc96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:59:10.022153 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-1435cd3b-0942-437c-9119-480b5ab3fc96 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:59:10.023381 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-1435cd3b-0942-437c-9119-480b5ab3fc96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:59:10.031175 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1381/2786] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:59:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:59:10.035914 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02150802-6678-417d-88f7-4dc406374af7 None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:59:10.045786 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02150802-6678-417d-88f7-4dc406374af7 None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:59:10.050482 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbaae0a-de80-4c60-91dc-c7cc227c3e1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:59:10.051130 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbaae0a-de80-4c60-91dc-c7cc227c3e1e admin admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:59:10.051442 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbaae0a-de80-4c60-91dc-c7cc227c3e1e admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:59:10.051721 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbaae0a-de80-4c60-91dc-c7cc227c3e1e admin admin] PATH_INFO: `/v3/domains` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:59:10.052419 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbaae0a-de80-4c60-91dc-c7cc227c3e1e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86827) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:59:10.052726 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbaae0a-de80-4c60-91dc-c7cc227c3e1e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:59:10.054359 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbaae0a-de80-4c60-91dc-c7cc227c3e1e admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:59:10.065843 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1406/2787] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Jul 27 10:59:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 10:59:10.069951 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1435cd3b-0942-437c-9119-480b5ab3fc96 None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:59:10.078998 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1435cd3b-0942-437c-9119-480b5ab3fc96 None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:59:10.084113 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313e881c-8a7e-4f24-a5d0-4a77a252f108 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:59:10.084718 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313e881c-8a7e-4f24-a5d0-4a77a252f108 admin admin] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:59:10.085001 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313e881c-8a7e-4f24-a5d0-4a77a252f108 admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:59:10.085271 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313e881c-8a7e-4f24-a5d0-4a77a252f108 admin admin] PATH_INFO: `/v3/projects` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:59:10.085933 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313e881c-8a7e-4f24-a5d0-4a77a252f108 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:59:10.087851 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313e881c-8a7e-4f24-a5d0-4a77a252f108 admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:59:10.114682 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1382/2788] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Jul 27 10:59:10 2022] POST /identity/v3/projects => generated 387 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 10:59:10.118736 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbaae0a-de80-4c60-91dc-c7cc227c3e1e admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:59:10.128921 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbaae0a-de80-4c60-91dc-c7cc227c3e1e admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:59:10.185697 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c5f3bd-76d0-4f0f-b2ea-f5b075c4f3c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:59:10.186315 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c5f3bd-76d0-4f0f-b2ea-f5b075c4f3c6 admin admin] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:59:10.186617 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c5f3bd-76d0-4f0f-b2ea-f5b075c4f3c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:59:10.186905 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c5f3bd-76d0-4f0f-b2ea-f5b075c4f3c6 admin admin] PATH_INFO: `/v3/users` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:59:10.187570 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c5f3bd-76d0-4f0f-b2ea-f5b075c4f3c6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:59:10.189804 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c5f3bd-76d0-4f0f-b2ea-f5b075c4f3c6 admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:59:10.211009 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1407/2789] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Jul 27 10:59:10 2022] POST /identity/v3/users => generated 294 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 10:59:10.215442 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313e881c-8a7e-4f24-a5d0-4a77a252f108 admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:59:10.226126 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313e881c-8a7e-4f24-a5d0-4a77a252f108 admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:59:10.230709 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76c0e2d-0652-463f-9bbf-725333781a2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:59:10.231347 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76c0e2d-0652-463f-9bbf-725333781a2e admin admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:59:10.231629 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76c0e2d-0652-463f-9bbf-725333781a2e admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:59:10.231903 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76c0e2d-0652-463f-9bbf-725333781a2e admin admin] PATH_INFO: `/v3/roles` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:59:10.232591 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76c0e2d-0652-463f-9bbf-725333781a2e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:59:10.234184 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76c0e2d-0652-463f-9bbf-725333781a2e admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:59:10.244019 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1383/2790] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 10:59:10 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 10:59:10.248105 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c5f3bd-76d0-4f0f-b2ea-f5b075c4f3c6 admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:59:10.258436 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c5f3bd-76d0-4f0f-b2ea-f5b075c4f3c6 admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:59:10.263010 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01796d34-e234-405b-8492-b0ee8095f3b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:59:10.263621 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01796d34-e234-405b-8492-b0ee8095f3b8 admin admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:59:10.263902 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01796d34-e234-405b-8492-b0ee8095f3b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:59:10.264176 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01796d34-e234-405b-8492-b0ee8095f3b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:59:10.264857 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01796d34-e234-405b-8492-b0ee8095f3b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:59:10.266431 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01796d34-e234-405b-8492-b0ee8095f3b8 admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:59:10.277206 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1408/2791] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 10:59:10 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 10:59:10.281186 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76c0e2d-0652-463f-9bbf-725333781a2e admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:59:10.290862 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76c0e2d-0652-463f-9bbf-725333781a2e admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:59:10.295397 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54bf22a-f5b8-48c0-8f02-12b1a883b7af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:59:10.296040 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54bf22a-f5b8-48c0-8f02-12b1a883b7af admin admin] REQUEST_METHOD: `PUT` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:59:10.296318 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54bf22a-f5b8-48c0-8f02-12b1a883b7af admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:59:10.296587 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54bf22a-f5b8-48c0-8f02-12b1a883b7af admin admin] PATH_INFO: `/v3/projects/9a8ea03db36c4881af5c03f1cdd563a1/users/459acdeceeb542fba53cdd712c6d6c81/roles/4c9fc0b3baed441da2126be41cc6b04a` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:59:10.310875 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54bf22a-f5b8-48c0-8f02-12b1a883b7af admin admin] RBAC: Authorizing `identity:create_grant(project_id=9a8ea03db36c4881af5c03f1cdd563a1, user_id=459acdeceeb542fba53cdd712c6d6c81, role_id=4c9fc0b3baed441da2126be41cc6b04a)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:59:10.313503 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54bf22a-f5b8-48c0-8f02-12b1a883b7af admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:59:10.326527 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1384/2792] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 10:59:10 2022] PUT /identity/v3/projects/9a8ea03db36c4881af5c03f1cdd563a1/users/459acdeceeb542fba53cdd712c6d6c81/roles/4c9fc0b3baed441da2126be41cc6b04a => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 10:59:10.336817 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185cac5e-159d-4410-9a4c-5d17de74b365 None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:59:10.337113 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185cac5e-159d-4410-9a4c-5d17de74b365 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:59:10.337398 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185cac5e-159d-4410-9a4c-5d17de74b365 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:59:10.364628 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-185cac5e-159d-4410-9a4c-5d17de74b365 None None] MFA Rules not processed for user `459acdeceeb542fba53cdd712c6d6c81`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:59:10.365836 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-185cac5e-159d-4410-9a4c-5d17de74b365 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:59:10.407955 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:59:10.431403 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1409/2793] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:59:10 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:59:10.435814 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54bf22a-f5b8-48c0-8f02-12b1a883b7af admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:59:10.479921 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:59:10.499187 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54bf22a-f5b8-48c0-8f02-12b1a883b7af admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:59:10.504401 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a756dcba-002f-4f8d-820b-a17bb04bae58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:59:10.505052 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a756dcba-002f-4f8d-820b-a17bb04bae58 admin admin] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:59:10.505337 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a756dcba-002f-4f8d-820b-a17bb04bae58 admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:59:10.505611 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a756dcba-002f-4f8d-820b-a17bb04bae58 admin admin] PATH_INFO: `/v3/projects` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:59:10.506270 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a756dcba-002f-4f8d-820b-a17bb04bae58 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:59:10.508269 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a756dcba-002f-4f8d-820b-a17bb04bae58 admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:59:10.539284 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1385/2794] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Jul 27 10:59:10 2022] POST /identity/v3/projects => generated 387 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 10:59:10.543428 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185cac5e-159d-4410-9a4c-5d17de74b365 None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:59:10.554275 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185cac5e-159d-4410-9a4c-5d17de74b365 None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:59:10.610742 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d093418-12fb-4a76-83f4-251a33f554fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:59:10.611657 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d093418-12fb-4a76-83f4-251a33f554fb admin admin] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:59:10.611935 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d093418-12fb-4a76-83f4-251a33f554fb admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:59:10.612204 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d093418-12fb-4a76-83f4-251a33f554fb admin admin] PATH_INFO: `/v3/users` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:59:10.612855 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d093418-12fb-4a76-83f4-251a33f554fb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:59:10.615303 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d093418-12fb-4a76-83f4-251a33f554fb admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:59:10.636785 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1410/2795] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Jul 27 10:59:10 2022] POST /identity/v3/users => generated 300 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 10:59:10.640819 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a756dcba-002f-4f8d-820b-a17bb04bae58 admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:59:10.651142 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a756dcba-002f-4f8d-820b-a17bb04bae58 admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:59:10.656073 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba9812a-d40f-4aa3-ba4b-0b0d2aff6f71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:59:10.656711 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba9812a-d40f-4aa3-ba4b-0b0d2aff6f71 admin admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:59:10.656996 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba9812a-d40f-4aa3-ba4b-0b0d2aff6f71 admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:59:10.657283 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba9812a-d40f-4aa3-ba4b-0b0d2aff6f71 admin admin] PATH_INFO: `/v3/roles` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:59:10.657979 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba9812a-d40f-4aa3-ba4b-0b0d2aff6f71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:59:10.659641 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba9812a-d40f-4aa3-ba4b-0b0d2aff6f71 admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:59:10.670157 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1386/2796] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 10:59:10 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 10:59:10.674203 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d093418-12fb-4a76-83f4-251a33f554fb admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:59:10.684578 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d093418-12fb-4a76-83f4-251a33f554fb admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:59:10.689155 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe816fe-54ba-457a-a55a-9e6d8dae4852 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:59:10.689807 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe816fe-54ba-457a-a55a-9e6d8dae4852 admin admin] REQUEST_METHOD: `PUT` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:59:10.690089 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe816fe-54ba-457a-a55a-9e6d8dae4852 admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:59:10.690365 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe816fe-54ba-457a-a55a-9e6d8dae4852 admin admin] PATH_INFO: `/v3/projects/65f6e66528cf4a9cacbf2b6018861339/users/71db1644ff69415aa0d98db4b4e4d4d7/roles/21156560eb6b436bb567a40b4976f3f4` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:59:10.705378 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe816fe-54ba-457a-a55a-9e6d8dae4852 admin admin] RBAC: Authorizing `identity:create_grant(project_id=65f6e66528cf4a9cacbf2b6018861339, user_id=71db1644ff69415aa0d98db4b4e4d4d7, role_id=21156560eb6b436bb567a40b4976f3f4)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:59:10.708083 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe816fe-54ba-457a-a55a-9e6d8dae4852 admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:59:10.721133 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1411/2797] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 10:59:10 2022] PUT /identity/v3/projects/65f6e66528cf4a9cacbf2b6018861339/users/71db1644ff69415aa0d98db4b4e4d4d7/roles/21156560eb6b436bb567a40b4976f3f4 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:59:10.725259 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba9812a-d40f-4aa3-ba4b-0b0d2aff6f71 admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:59:10.736222 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba9812a-d40f-4aa3-ba4b-0b0d2aff6f71 admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:59:10.794046 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca48e9c-260f-403c-b1ef-b60e13cfa346 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:59:10.794692 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca48e9c-260f-403c-b1ef-b60e13cfa346 admin admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:59:10.794894 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca48e9c-260f-403c-b1ef-b60e13cfa346 admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:59:10.795194 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca48e9c-260f-403c-b1ef-b60e13cfa346 admin admin] PATH_INFO: `/v3/roles` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:59:10.795915 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca48e9c-260f-403c-b1ef-b60e13cfa346 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:59:10.797528 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca48e9c-260f-403c-b1ef-b60e13cfa346 admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:59:10.807543 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1387/2798] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 10:59:10 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 10:59:10.811809 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe816fe-54ba-457a-a55a-9e6d8dae4852 admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:59:10.822424 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe816fe-54ba-457a-a55a-9e6d8dae4852 admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:59:10.826981 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5586da-5f98-434a-b926-4fbc535ef2eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:59:10.827659 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5586da-5f98-434a-b926-4fbc535ef2eb admin admin] REQUEST_METHOD: `PUT` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:59:10.827937 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5586da-5f98-434a-b926-4fbc535ef2eb admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:59:10.828208 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5586da-5f98-434a-b926-4fbc535ef2eb admin admin] PATH_INFO: `/v3/projects/65f6e66528cf4a9cacbf2b6018861339/users/71db1644ff69415aa0d98db4b4e4d4d7/roles/21156560eb6b436bb567a40b4976f3f4` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:59:10.830123 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5586da-5f98-434a-b926-4fbc535ef2eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=65f6e66528cf4a9cacbf2b6018861339, user_id=71db1644ff69415aa0d98db4b4e4d4d7, role_id=21156560eb6b436bb567a40b4976f3f4)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:59:10.832762 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5586da-5f98-434a-b926-4fbc535ef2eb admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:59:10.844508 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1412/2799] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 10:59:10 2022] PUT /identity/v3/projects/65f6e66528cf4a9cacbf2b6018861339/users/71db1644ff69415aa0d98db4b4e4d4d7/roles/21156560eb6b436bb567a40b4976f3f4 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:59:10.858767 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019ddbd9-9274-445a-b88b-b2ad019b7e9c None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:59:10.859059 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019ddbd9-9274-445a-b88b-b2ad019b7e9c None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:59:10.859336 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019ddbd9-9274-445a-b88b-b2ad019b7e9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:59:10.887768 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-019ddbd9-9274-445a-b88b-b2ad019b7e9c None None] MFA Rules not processed for user `71db1644ff69415aa0d98db4b4e4d4d7`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:59:10.888970 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-019ddbd9-9274-445a-b88b-b2ad019b7e9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:59:10.958946 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1388/2800] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:59:10 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:59:10.970352 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5586da-5f98-434a-b926-4fbc535ef2eb admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:59:10.972140 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-be5586da-5f98-434a-b926-4fbc535ef2eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:59:11.029916 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5586da-5f98-434a-b926-4fbc535ef2eb admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:59:11.033962 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae60bfd9-5e9e-40bd-995d-3984e50eb756 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '864d35f268cf45148756109759bb2743', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:59:11.034580 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae60bfd9-5e9e-40bd-995d-3984e50eb756 service cinder] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:59:11.034894 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae60bfd9-5e9e-40bd-995d-3984e50eb756 service cinder] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:59:11.035669 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae60bfd9-5e9e-40bd-995d-3984e50eb756 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:59:11.044696 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae60bfd9-5e9e-40bd-995d-3984e50eb756 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:59:11.046075 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae60bfd9-5e9e-40bd-995d-3984e50eb756 service cinder] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:59:11.056772 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1413/2801] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:59:10 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 10:59:11.613144 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019ddbd9-9274-445a-b88b-b2ad019b7e9c None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:59:11.653696 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:59:11.676355 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019ddbd9-9274-445a-b88b-b2ad019b7e9c None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:59:11.680932 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9056d24b-9f21-4bce-ac59-ae004162711a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:59:11.681578 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9056d24b-9f21-4bce-ac59-ae004162711a admin admin] REQUEST_METHOD: `DELETE` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:59:11.681863 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9056d24b-9f21-4bce-ac59-ae004162711a admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:59:11.682173 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9056d24b-9f21-4bce-ac59-ae004162711a admin admin] PATH_INFO: `/v3/users/459acdeceeb542fba53cdd712c6d6c81` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:59:11.683362 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9056d24b-9f21-4bce-ac59-ae004162711a admin admin] RBAC: Authorizing `identity:delete_user(user_id=459acdeceeb542fba53cdd712c6d6c81)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:59:11.685320 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9056d24b-9f21-4bce-ac59-ae004162711a admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:59:11.732719 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-9056d24b-9f21-4bce-ac59-ae004162711a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '459acdeceeb542fba53cdd712c6d6c81'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:59:11.739494 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-9056d24b-9f21-4bce-ac59-ae004162711a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '459acdeceeb542fba53cdd712c6d6c81'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:59:11.746822 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-9056d24b-9f21-4bce-ac59-ae004162711a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '459acdeceeb542fba53cdd712c6d6c81'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:59:11.747420 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-9056d24b-9f21-4bce-ac59-ae004162711a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '459acdeceeb542fba53cdd712c6d6c81'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:59:11.748092 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-9056d24b-9f21-4bce-ac59-ae004162711a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '459acdeceeb542fba53cdd712c6d6c81'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:59:11.766338 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1389/2802] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Jul 27 10:59:11 2022] DELETE /identity/v3/users/459acdeceeb542fba53cdd712c6d6c81 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:59:11.771508 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4666602a-2d2a-45c9-ba77-f52ca42ca614 None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:59:11.771982 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4666602a-2d2a-45c9-ba77-f52ca42ca614 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:59:11.772101 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4666602a-2d2a-45c9-ba77-f52ca42ca614 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:59:11.790430 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-4666602a-2d2a-45c9-ba77-f52ca42ca614 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:59:11.791670 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-4666602a-2d2a-45c9-ba77-f52ca42ca614 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:59:11.823768 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:59:11.851767 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1414/2803] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:59:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:59:11.861449 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9056d24b-9f21-4bce-ac59-ae004162711a admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:59:11.862889 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-9056d24b-9f21-4bce-ac59-ae004162711a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:59:11.921726 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9056d24b-9f21-4bce-ac59-ae004162711a admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:59:11.925335 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a37955-926b-4947-a28e-56975143cf3c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7922d0a1a7c4e808d0b2a80dca0da40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:59:11.925946 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a37955-926b-4947-a28e-56975143cf3c service neutron] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:59:11.926225 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a37955-926b-4947-a28e-56975143cf3c service neutron] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:59:11.926495 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a37955-926b-4947-a28e-56975143cf3c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:59:11.935204 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a37955-926b-4947-a28e-56975143cf3c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:59:11.936623 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a37955-926b-4947-a28e-56975143cf3c service neutron] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:59:11.946846 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1390/2804] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:59:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 10:59:12.218190 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4666602a-2d2a-45c9-ba77-f52ca42ca614 None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:59:12.219951 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-4666602a-2d2a-45c9-ba77-f52ca42ca614 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:59:12.237569 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4666602a-2d2a-45c9-ba77-f52ca42ca614 None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:59:12.243710 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a94d006-942f-44a5-9fa8-cf13f3dec420 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:59:12.244414 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a94d006-942f-44a5-9fa8-cf13f3dec420 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:59:12.244699 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a94d006-942f-44a5-9fa8-cf13f3dec420 admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:59:12.244988 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a94d006-942f-44a5-9fa8-cf13f3dec420 admin admin] PATH_INFO: `/v3/projects/9a8ea03db36c4881af5c03f1cdd563a1` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:59:12.246373 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a94d006-942f-44a5-9fa8-cf13f3dec420 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9a8ea03db36c4881af5c03f1cdd563a1)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:59:12.248425 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a94d006-942f-44a5-9fa8-cf13f3dec420 admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:59:12.294224 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-8a94d006-942f-44a5-9fa8-cf13f3dec420 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9a8ea03db36c4881af5c03f1cdd563a1', 'request_id': 'req-8a94d006-942f-44a5-9fa8-cf13f3dec420'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:59:12.301221 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-8a94d006-942f-44a5-9fa8-cf13f3dec420 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9a8ea03db36c4881af5c03f1cdd563a1', 'request_id': 'req-8a94d006-942f-44a5-9fa8-cf13f3dec420'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:59:12.308093 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-8a94d006-942f-44a5-9fa8-cf13f3dec420 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9a8ea03db36c4881af5c03f1cdd563a1', 'request_id': 'req-8a94d006-942f-44a5-9fa8-cf13f3dec420'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:59:12.319468 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-8a94d006-942f-44a5-9fa8-cf13f3dec420 admin admin] The token cache is being invalidate because project 9a8ea03db36c4881af5c03f1cdd563a1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86827) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 10:59:12.319757 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-8a94d006-942f-44a5-9fa8-cf13f3dec420 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:59:12.320379 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-8a94d006-942f-44a5-9fa8-cf13f3dec420 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:59:12.321893 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1415/2805] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Jul 27 10:59:12 2022] DELETE /identity/v3/projects/9a8ea03db36c4881af5c03f1cdd563a1 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:59:12.325896 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a37955-926b-4947-a28e-56975143cf3c service neutron] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:59:12.335407 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a37955-926b-4947-a28e-56975143cf3c service neutron] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:59:12.391176 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87f9080-d798-42f6-8762-2d2c834e79c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:59:12.391809 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87f9080-d798-42f6-8762-2d2c834e79c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:59:12.392089 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87f9080-d798-42f6-8762-2d2c834e79c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:59:12.392363 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87f9080-d798-42f6-8762-2d2c834e79c4 admin admin] PATH_INFO: `/v3/users/71db1644ff69415aa0d98db4b4e4d4d7` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:59:12.393513 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87f9080-d798-42f6-8762-2d2c834e79c4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=71db1644ff69415aa0d98db4b4e4d4d7)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:59:12.395493 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87f9080-d798-42f6-8762-2d2c834e79c4 admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:59:12.396925 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:59:12.442168 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-d87f9080-d798-42f6-8762-2d2c834e79c4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '71db1644ff69415aa0d98db4b4e4d4d7'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:59:12.448929 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-d87f9080-d798-42f6-8762-2d2c834e79c4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '71db1644ff69415aa0d98db4b4e4d4d7'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:59:12.455799 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-d87f9080-d798-42f6-8762-2d2c834e79c4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '71db1644ff69415aa0d98db4b4e4d4d7'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:59:12.456385 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-d87f9080-d798-42f6-8762-2d2c834e79c4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '71db1644ff69415aa0d98db4b4e4d4d7'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:59:12.457036 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-d87f9080-d798-42f6-8762-2d2c834e79c4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '71db1644ff69415aa0d98db4b4e4d4d7'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:59:12.474354 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1391/2806] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Jul 27 10:59:12 2022] DELETE /identity/v3/users/71db1644ff69415aa0d98db4b4e4d4d7 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:59:12.742048 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a94d006-942f-44a5-9fa8-cf13f3dec420 admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:59:12.743493 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-8a94d006-942f-44a5-9fa8-cf13f3dec420 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:59:12.759190 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a94d006-942f-44a5-9fa8-cf13f3dec420 admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:59:12.816400 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456ff55a-84ff-44f4-bc5f-574fc2657172 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:59:12.817061 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456ff55a-84ff-44f4-bc5f-574fc2657172 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:59:12.817311 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456ff55a-84ff-44f4-bc5f-574fc2657172 admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:59:12.817585 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456ff55a-84ff-44f4-bc5f-574fc2657172 admin admin] PATH_INFO: `/v3/projects/65f6e66528cf4a9cacbf2b6018861339` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:59:12.818774 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456ff55a-84ff-44f4-bc5f-574fc2657172 admin admin] RBAC: Authorizing `identity:delete_project(project_id=65f6e66528cf4a9cacbf2b6018861339)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:59:12.820674 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456ff55a-84ff-44f4-bc5f-574fc2657172 admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:59:12.870319 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-456ff55a-84ff-44f4-bc5f-574fc2657172 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '65f6e66528cf4a9cacbf2b6018861339', 'request_id': 'req-456ff55a-84ff-44f4-bc5f-574fc2657172'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:59:12.876854 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-456ff55a-84ff-44f4-bc5f-574fc2657172 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '65f6e66528cf4a9cacbf2b6018861339', 'request_id': 'req-456ff55a-84ff-44f4-bc5f-574fc2657172'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:59:12.884387 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-456ff55a-84ff-44f4-bc5f-574fc2657172 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '65f6e66528cf4a9cacbf2b6018861339', 'request_id': 'req-456ff55a-84ff-44f4-bc5f-574fc2657172'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:59:12.886437 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:59:12.896701 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-456ff55a-84ff-44f4-bc5f-574fc2657172 admin admin] The token cache is being invalidate because project 65f6e66528cf4a9cacbf2b6018861339 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86827) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 10:59:12.896995 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-456ff55a-84ff-44f4-bc5f-574fc2657172 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:59:12.897615 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-456ff55a-84ff-44f4-bc5f-574fc2657172 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:59:12.899150 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1416/2807] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Jul 27 10:59:12 2022] DELETE /identity/v3/projects/65f6e66528cf4a9cacbf2b6018861339 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:59:12.904080 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2aab252-dc42-4571-a8d0-7045c4f2daed None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:59:12.904556 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2aab252-dc42-4571-a8d0-7045c4f2daed None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:59:12.904678 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2aab252-dc42-4571-a8d0-7045c4f2daed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:59:12.924325 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-f2aab252-dc42-4571-a8d0-7045c4f2daed None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:59:12.925544 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-f2aab252-dc42-4571-a8d0-7045c4f2daed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:59:12.983814 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1392/2808] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:59:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:59:12.991459 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf16e05-eb51-49d8-ac03-a37e3dacadc0 None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:59:12.991747 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf16e05-eb51-49d8-ac03-a37e3dacadc0 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:59:12.992023 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf16e05-eb51-49d8-ac03-a37e3dacadc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:59:13.011238 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-5bf16e05-eb51-49d8-ac03-a37e3dacadc0 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:59:13.012435 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-5bf16e05-eb51-49d8-ac03-a37e3dacadc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:59:13.020634 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1417/2809] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:59:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:59:13.024673 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2aab252-dc42-4571-a8d0-7045c4f2daed None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:59:13.034298 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2aab252-dc42-4571-a8d0-7045c4f2daed None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:59:13.039510 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f8d578-e3ac-4f17-b3da-c7d48598f7fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:59:13.040118 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f8d578-e3ac-4f17-b3da-c7d48598f7fb admin admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:59:13.040400 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f8d578-e3ac-4f17-b3da-c7d48598f7fb admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:59:13.040675 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f8d578-e3ac-4f17-b3da-c7d48598f7fb admin admin] PATH_INFO: `/v3/domains` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:59:13.041373 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f8d578-e3ac-4f17-b3da-c7d48598f7fb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86828) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:59:13.041677 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f8d578-e3ac-4f17-b3da-c7d48598f7fb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:59:13.043335 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f8d578-e3ac-4f17-b3da-c7d48598f7fb admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:59:13.054657 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1393/2810] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Jul 27 10:59:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:59:13.058789 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf16e05-eb51-49d8-ac03-a37e3dacadc0 None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:59:13.067793 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf16e05-eb51-49d8-ac03-a37e3dacadc0 None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:59:13.072359 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf879bdd-ab05-4a56-aed6-e2a6bc9b1150 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:59:13.072953 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf879bdd-ab05-4a56-aed6-e2a6bc9b1150 admin admin] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:59:13.073233 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf879bdd-ab05-4a56-aed6-e2a6bc9b1150 admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:59:13.073503 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf879bdd-ab05-4a56-aed6-e2a6bc9b1150 admin admin] PATH_INFO: `/v3/projects` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:59:13.074169 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf879bdd-ab05-4a56-aed6-e2a6bc9b1150 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:59:13.076070 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf879bdd-ab05-4a56-aed6-e2a6bc9b1150 admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:59:13.103031 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1418/2811] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Jul 27 10:59:13 2022] POST /identity/v3/projects => generated 385 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 10:59:13.108194 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f8d578-e3ac-4f17-b3da-c7d48598f7fb admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:59:13.117950 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f8d578-e3ac-4f17-b3da-c7d48598f7fb admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:59:13.172945 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa0c284-2307-4111-9049-18cebff8f451 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:59:13.173521 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa0c284-2307-4111-9049-18cebff8f451 admin admin] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:59:13.173791 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa0c284-2307-4111-9049-18cebff8f451 admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:59:13.174063 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa0c284-2307-4111-9049-18cebff8f451 admin admin] PATH_INFO: `/v3/users` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:59:13.174748 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa0c284-2307-4111-9049-18cebff8f451 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:59:13.176640 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa0c284-2307-4111-9049-18cebff8f451 admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:59:13.187546 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:59:13.201320 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1394/2812] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Jul 27 10:59:13 2022] POST /identity/v3/users => generated 293 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 10:59:13.205434 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf879bdd-ab05-4a56-aed6-e2a6bc9b1150 admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:59:13.216012 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf879bdd-ab05-4a56-aed6-e2a6bc9b1150 admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:59:13.221051 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ee8338-165f-4192-8e63-864af7022dd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:59:13.221742 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ee8338-165f-4192-8e63-864af7022dd6 admin admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:59:13.222070 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ee8338-165f-4192-8e63-864af7022dd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:59:13.222397 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ee8338-165f-4192-8e63-864af7022dd6 admin admin] PATH_INFO: `/v3/roles` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:59:13.223782 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ee8338-165f-4192-8e63-864af7022dd6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:59:13.225370 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ee8338-165f-4192-8e63-864af7022dd6 admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:59:13.235266 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1419/2813] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 10:59:13 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 10:59:13.239127 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa0c284-2307-4111-9049-18cebff8f451 admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:59:13.249290 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa0c284-2307-4111-9049-18cebff8f451 admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:59:13.254923 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add7c41b-0589-453f-8c46-ed529f96c8d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:59:13.255529 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add7c41b-0589-453f-8c46-ed529f96c8d7 admin admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:59:13.255814 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add7c41b-0589-453f-8c46-ed529f96c8d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:59:13.256089 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add7c41b-0589-453f-8c46-ed529f96c8d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:59:13.256779 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add7c41b-0589-453f-8c46-ed529f96c8d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:59:13.258367 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add7c41b-0589-453f-8c46-ed529f96c8d7 admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:59:13.268911 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1395/2814] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 10:59:13 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 10:59:13.273662 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ee8338-165f-4192-8e63-864af7022dd6 admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:59:13.283536 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ee8338-165f-4192-8e63-864af7022dd6 admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:59:13.288247 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3465aa-506e-4916-b706-6a40b021ae63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:59:13.288925 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3465aa-506e-4916-b706-6a40b021ae63 admin admin] REQUEST_METHOD: `PUT` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:59:13.289211 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3465aa-506e-4916-b706-6a40b021ae63 admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:59:13.289490 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3465aa-506e-4916-b706-6a40b021ae63 admin admin] PATH_INFO: `/v3/projects/92cb2580902e4bf4b320c93d3d1c5de8/users/932462442d82436e86a062c212563b76/roles/4c9fc0b3baed441da2126be41cc6b04a` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:59:13.303913 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3465aa-506e-4916-b706-6a40b021ae63 admin admin] RBAC: Authorizing `identity:create_grant(project_id=92cb2580902e4bf4b320c93d3d1c5de8, user_id=932462442d82436e86a062c212563b76, role_id=4c9fc0b3baed441da2126be41cc6b04a)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:59:13.306651 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3465aa-506e-4916-b706-6a40b021ae63 admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:59:13.319857 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1420/2815] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 10:59:13 2022] PUT /identity/v3/projects/92cb2580902e4bf4b320c93d3d1c5de8/users/932462442d82436e86a062c212563b76/roles/4c9fc0b3baed441da2126be41cc6b04a => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:59:13.330091 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0e8b87-015f-4eb4-ad0a-61275d0a566b None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:59:13.330435 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0e8b87-015f-4eb4-ad0a-61275d0a566b None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:59:13.330753 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0e8b87-015f-4eb4-ad0a-61275d0a566b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:59:13.358361 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-de0e8b87-015f-4eb4-ad0a-61275d0a566b None None] MFA Rules not processed for user `932462442d82436e86a062c212563b76`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:59:13.359676 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-de0e8b87-015f-4eb4-ad0a-61275d0a566b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:59:13.422589 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1396/2816] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:59:13 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:59:13.433607 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3465aa-506e-4916-b706-6a40b021ae63 admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:59:13.435325 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-ab3465aa-506e-4916-b706-6a40b021ae63 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:59:13.492369 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3465aa-506e-4916-b706-6a40b021ae63 admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:59:13.496230 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9526b316-f84c-4ac3-8b3d-488fead8b217 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '864d35f268cf45148756109759bb2743', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:59:13.496838 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9526b316-f84c-4ac3-8b3d-488fead8b217 service cinder] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:59:13.497121 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9526b316-f84c-4ac3-8b3d-488fead8b217 service cinder] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:59:13.497393 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9526b316-f84c-4ac3-8b3d-488fead8b217 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:59:13.506107 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9526b316-f84c-4ac3-8b3d-488fead8b217 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:59:13.507545 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9526b316-f84c-4ac3-8b3d-488fead8b217 service cinder] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:59:13.516250 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1421/2817] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:59:13 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 10:59:13.536143 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0e8b87-015f-4eb4-ad0a-61275d0a566b None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:59:13.597602 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0e8b87-015f-4eb4-ad0a-61275d0a566b None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:59:13.602204 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f81dc46-76e3-4c99-8e8c-1d4f4ebdd12a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:59:13.602882 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f81dc46-76e3-4c99-8e8c-1d4f4ebdd12a admin admin] REQUEST_METHOD: `DELETE` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:59:13.603168 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f81dc46-76e3-4c99-8e8c-1d4f4ebdd12a admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:59:13.603446 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f81dc46-76e3-4c99-8e8c-1d4f4ebdd12a admin admin] PATH_INFO: `/v3/users/932462442d82436e86a062c212563b76` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:59:13.604622 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f81dc46-76e3-4c99-8e8c-1d4f4ebdd12a admin admin] RBAC: Authorizing `identity:delete_user(user_id=932462442d82436e86a062c212563b76)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:59:13.606544 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f81dc46-76e3-4c99-8e8c-1d4f4ebdd12a admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:59:13.653308 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-0f81dc46-76e3-4c99-8e8c-1d4f4ebdd12a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '932462442d82436e86a062c212563b76'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:59:13.660301 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-0f81dc46-76e3-4c99-8e8c-1d4f4ebdd12a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '932462442d82436e86a062c212563b76'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:59:13.667656 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-0f81dc46-76e3-4c99-8e8c-1d4f4ebdd12a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '932462442d82436e86a062c212563b76'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:59:13.668286 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-0f81dc46-76e3-4c99-8e8c-1d4f4ebdd12a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '932462442d82436e86a062c212563b76'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:59:13.668907 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-0f81dc46-76e3-4c99-8e8c-1d4f4ebdd12a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '932462442d82436e86a062c212563b76'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:59:13.679435 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:59:13.688514 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1397/2818] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Jul 27 10:59:13 2022] DELETE /identity/v3/users/932462442d82436e86a062c212563b76 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:59:13.693610 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e027498-d62d-4a5c-9e33-fd478c751f73 None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:59:13.693901 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e027498-d62d-4a5c-9e33-fd478c751f73 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:59:13.694180 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e027498-d62d-4a5c-9e33-fd478c751f73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:59:13.712485 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-1e027498-d62d-4a5c-9e33-fd478c751f73 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:59:13.713680 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-1e027498-d62d-4a5c-9e33-fd478c751f73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:59:13.772603 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1422/2819] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:59:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:59:13.782177 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f81dc46-76e3-4c99-8e8c-1d4f4ebdd12a admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:59:13.783586 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-0f81dc46-76e3-4c99-8e8c-1d4f4ebdd12a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:59:13.839377 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f81dc46-76e3-4c99-8e8c-1d4f4ebdd12a admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:59:13.842964 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc0e3ce-3039-45e4-bdc7-ae89a3048da1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7922d0a1a7c4e808d0b2a80dca0da40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:59:13.843576 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc0e3ce-3039-45e4-bdc7-ae89a3048da1 service neutron] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:59:13.843855 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc0e3ce-3039-45e4-bdc7-ae89a3048da1 service neutron] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:59:13.844127 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc0e3ce-3039-45e4-bdc7-ae89a3048da1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:59:13.852950 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc0e3ce-3039-45e4-bdc7-ae89a3048da1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:59:13.854361 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc0e3ce-3039-45e4-bdc7-ae89a3048da1 service neutron] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:59:13.864096 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1398/2820] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:59:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 10:59:14.130773 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e027498-d62d-4a5c-9e33-fd478c751f73 None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:59:14.132293 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-1e027498-d62d-4a5c-9e33-fd478c751f73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:59:14.145912 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e027498-d62d-4a5c-9e33-fd478c751f73 None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:59:14.151375 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e244d03a-1dc5-43a3-9c37-35bd7ae7ae0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:59:14.152198 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e244d03a-1dc5-43a3-9c37-35bd7ae7ae0b admin admin] REQUEST_METHOD: `DELETE` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:59:14.152907 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e244d03a-1dc5-43a3-9c37-35bd7ae7ae0b admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:59:14.153203 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e244d03a-1dc5-43a3-9c37-35bd7ae7ae0b admin admin] PATH_INFO: `/v3/projects/92cb2580902e4bf4b320c93d3d1c5de8` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:59:14.154421 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e244d03a-1dc5-43a3-9c37-35bd7ae7ae0b admin admin] RBAC: Authorizing `identity:delete_project(project_id=92cb2580902e4bf4b320c93d3d1c5de8)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:59:14.156376 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e244d03a-1dc5-43a3-9c37-35bd7ae7ae0b admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:59:14.188656 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=96,keystone:DELETE=2,keystone:INSERT=2 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:59:14.205879 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-e244d03a-1dc5-43a3-9c37-35bd7ae7ae0b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '92cb2580902e4bf4b320c93d3d1c5de8', 'request_id': 'req-e244d03a-1dc5-43a3-9c37-35bd7ae7ae0b'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:59:14.212018 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-e244d03a-1dc5-43a3-9c37-35bd7ae7ae0b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '92cb2580902e4bf4b320c93d3d1c5de8', 'request_id': 'req-e244d03a-1dc5-43a3-9c37-35bd7ae7ae0b'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:59:14.219548 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-e244d03a-1dc5-43a3-9c37-35bd7ae7ae0b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '92cb2580902e4bf4b320c93d3d1c5de8', 'request_id': 'req-e244d03a-1dc5-43a3-9c37-35bd7ae7ae0b'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:59:14.230515 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-e244d03a-1dc5-43a3-9c37-35bd7ae7ae0b admin admin] The token cache is being invalidate because project 92cb2580902e4bf4b320c93d3d1c5de8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86827) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 10:59:14.230837 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-e244d03a-1dc5-43a3-9c37-35bd7ae7ae0b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:59:14.231461 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-e244d03a-1dc5-43a3-9c37-35bd7ae7ae0b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:59:14.232942 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1423/2821] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Jul 27 10:59:14 2022] DELETE /identity/v3/projects/92cb2580902e4bf4b320c93d3d1c5de8 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:59:14.237899 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6217071b-a98a-4a4b-aa68-303a3c3b638e None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:59:14.238971 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6217071b-a98a-4a4b-aa68-303a3c3b638e None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:59:14.239136 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6217071b-a98a-4a4b-aa68-303a3c3b638e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:59:14.259594 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-6217071b-a98a-4a4b-aa68-303a3c3b638e None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:59:14.260786 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-6217071b-a98a-4a4b-aa68-303a3c3b638e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:59:14.317790 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1399/2822] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:59:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:59:14.326800 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8858c4-ecde-4bf1-a5b1-d25f58d51fd9 None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:59:14.327083 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8858c4-ecde-4bf1-a5b1-d25f58d51fd9 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:59:14.327353 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8858c4-ecde-4bf1-a5b1-d25f58d51fd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:59:14.346844 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-0f8858c4-ecde-4bf1-a5b1-d25f58d51fd9 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:59:14.348656 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-0f8858c4-ecde-4bf1-a5b1-d25f58d51fd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:59:14.356364 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1424/2823] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:59:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:59:14.360360 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6217071b-a98a-4a4b-aa68-303a3c3b638e None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:59:14.369883 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6217071b-a98a-4a4b-aa68-303a3c3b638e None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:59:14.374839 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a383d37d-8da2-4ad5-b44f-152f1685e912 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:59:14.375449 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a383d37d-8da2-4ad5-b44f-152f1685e912 admin admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:59:14.375735 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a383d37d-8da2-4ad5-b44f-152f1685e912 admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:59:14.376010 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a383d37d-8da2-4ad5-b44f-152f1685e912 admin admin] PATH_INFO: `/v3/domains` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:59:14.376703 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a383d37d-8da2-4ad5-b44f-152f1685e912 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86828) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:59:14.377015 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a383d37d-8da2-4ad5-b44f-152f1685e912 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:59:14.378625 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a383d37d-8da2-4ad5-b44f-152f1685e912 admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:59:14.389930 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1400/2824] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Jul 27 10:59:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:59:14.394010 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8858c4-ecde-4bf1-a5b1-d25f58d51fd9 None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:59:14.403605 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8858c4-ecde-4bf1-a5b1-d25f58d51fd9 None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:59:14.408113 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97f0f2b-33d8-400a-9399-55f0fc620da3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:59:14.408709 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97f0f2b-33d8-400a-9399-55f0fc620da3 admin admin] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:59:14.408986 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97f0f2b-33d8-400a-9399-55f0fc620da3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:59:14.409259 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97f0f2b-33d8-400a-9399-55f0fc620da3 admin admin] PATH_INFO: `/v3/projects` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:59:14.409916 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97f0f2b-33d8-400a-9399-55f0fc620da3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:59:14.412375 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97f0f2b-33d8-400a-9399-55f0fc620da3 admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:59:14.438742 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1425/2825] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Jul 27 10:59:14 2022] POST /identity/v3/projects => generated 373 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 10:59:14.443023 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a383d37d-8da2-4ad5-b44f-152f1685e912 admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:59:14.452802 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a383d37d-8da2-4ad5-b44f-152f1685e912 admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:59:14.507526 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3166cca-7181-45ab-ac60-fd9c7ec3f20b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:59:14.508055 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3166cca-7181-45ab-ac60-fd9c7ec3f20b admin admin] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:59:14.508333 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3166cca-7181-45ab-ac60-fd9c7ec3f20b admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:59:14.508607 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3166cca-7181-45ab-ac60-fd9c7ec3f20b admin admin] PATH_INFO: `/v3/users` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:59:14.509252 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3166cca-7181-45ab-ac60-fd9c7ec3f20b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:59:14.511185 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3166cca-7181-45ab-ac60-fd9c7ec3f20b admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:59:14.532000 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1401/2826] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Jul 27 10:59:14 2022] POST /identity/v3/users => generated 287 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 10:59:14.536715 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97f0f2b-33d8-400a-9399-55f0fc620da3 admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:59:14.547017 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97f0f2b-33d8-400a-9399-55f0fc620da3 admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:59:14.551800 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109abab8-e0e6-4a3d-9067-4350934b7d00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:59:14.552500 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109abab8-e0e6-4a3d-9067-4350934b7d00 admin admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:59:14.552780 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109abab8-e0e6-4a3d-9067-4350934b7d00 admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:59:14.554056 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109abab8-e0e6-4a3d-9067-4350934b7d00 admin admin] PATH_INFO: `/v3/roles` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:59:14.555219 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109abab8-e0e6-4a3d-9067-4350934b7d00 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:59:14.557583 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109abab8-e0e6-4a3d-9067-4350934b7d00 admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:59:14.567851 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1426/2827] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 10:59:14 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 10:59:14.572064 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3166cca-7181-45ab-ac60-fd9c7ec3f20b admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:59:14.581791 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3166cca-7181-45ab-ac60-fd9c7ec3f20b admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:59:14.586903 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d90cca1-f3b1-4b3a-ad5a-16e333253d23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:59:14.587507 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d90cca1-f3b1-4b3a-ad5a-16e333253d23 admin admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:59:14.587785 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d90cca1-f3b1-4b3a-ad5a-16e333253d23 admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:59:14.588056 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d90cca1-f3b1-4b3a-ad5a-16e333253d23 admin admin] PATH_INFO: `/v3/roles` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:59:14.588736 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d90cca1-f3b1-4b3a-ad5a-16e333253d23 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:59:14.590307 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d90cca1-f3b1-4b3a-ad5a-16e333253d23 admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:59:14.601160 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1402/2828] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 10:59:14 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 10:59:14.605271 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109abab8-e0e6-4a3d-9067-4350934b7d00 admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:59:14.615238 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109abab8-e0e6-4a3d-9067-4350934b7d00 admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:59:14.619878 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a67159-9247-4d15-9164-e666a5e25ee6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:59:14.620788 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a67159-9247-4d15-9164-e666a5e25ee6 admin admin] REQUEST_METHOD: `PUT` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:59:14.621074 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a67159-9247-4d15-9164-e666a5e25ee6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:59:14.621354 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a67159-9247-4d15-9164-e666a5e25ee6 admin admin] PATH_INFO: `/v3/projects/49a315768bd94707adeb4e9f0eb12a8c/users/5f6cb25e6a164de082783ad01b2714bc/roles/4c9fc0b3baed441da2126be41cc6b04a` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:59:14.635872 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a67159-9247-4d15-9164-e666a5e25ee6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=49a315768bd94707adeb4e9f0eb12a8c, user_id=5f6cb25e6a164de082783ad01b2714bc, role_id=4c9fc0b3baed441da2126be41cc6b04a)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:59:14.638515 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a67159-9247-4d15-9164-e666a5e25ee6 admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:59:14.651948 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1427/2829] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 10:59:14 2022] PUT /identity/v3/projects/49a315768bd94707adeb4e9f0eb12a8c/users/5f6cb25e6a164de082783ad01b2714bc/roles/4c9fc0b3baed441da2126be41cc6b04a => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 10:59:14.662669 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c68ffe-92eb-481f-913d-453d0053e641 None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:59:14.662965 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c68ffe-92eb-481f-913d-453d0053e641 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:59:14.663247 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c68ffe-92eb-481f-913d-453d0053e641 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:59:14.670783 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:59:14.693334 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-65c68ffe-92eb-481f-913d-453d0053e641 None None] MFA Rules not processed for user `5f6cb25e6a164de082783ad01b2714bc`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:59:14.694543 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-65c68ffe-92eb-481f-913d-453d0053e641 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:59:14.759795 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1403/2830] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:59:14 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 10:59:14.771005 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a67159-9247-4d15-9164-e666a5e25ee6 admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:59:14.772689 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-a1a67159-9247-4d15-9164-e666a5e25ee6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:59:14.828626 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a67159-9247-4d15-9164-e666a5e25ee6 admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:59:14.832983 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca800c1f-9444-45f6-a3d8-a1d200fe7b3a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '864d35f268cf45148756109759bb2743', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:59:14.833593 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca800c1f-9444-45f6-a3d8-a1d200fe7b3a service cinder] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:59:14.833876 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca800c1f-9444-45f6-a3d8-a1d200fe7b3a service cinder] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:59:14.834153 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca800c1f-9444-45f6-a3d8-a1d200fe7b3a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:59:14.842570 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca800c1f-9444-45f6-a3d8-a1d200fe7b3a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:59:14.844055 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca800c1f-9444-45f6-a3d8-a1d200fe7b3a service cinder] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:59:14.852796 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1428/2831] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:59:14 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 10:59:18.037005 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c68ffe-92eb-481f-913d-453d0053e641 None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:59:18.097969 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c68ffe-92eb-481f-913d-453d0053e641 None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:59:18.102460 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f7d69a-e726-418c-b176-a6428e1d714b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:59:18.103134 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f7d69a-e726-418c-b176-a6428e1d714b admin admin] REQUEST_METHOD: `DELETE` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:59:18.103421 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f7d69a-e726-418c-b176-a6428e1d714b admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:59:18.103695 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f7d69a-e726-418c-b176-a6428e1d714b admin admin] PATH_INFO: `/v3/users/5f6cb25e6a164de082783ad01b2714bc` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:59:18.104860 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f7d69a-e726-418c-b176-a6428e1d714b admin admin] RBAC: Authorizing `identity:delete_user(user_id=5f6cb25e6a164de082783ad01b2714bc)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:59:18.106911 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f7d69a-e726-418c-b176-a6428e1d714b admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:59:18.156212 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-69f7d69a-e726-418c-b176-a6428e1d714b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5f6cb25e6a164de082783ad01b2714bc'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:59:18.163623 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-69f7d69a-e726-418c-b176-a6428e1d714b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5f6cb25e6a164de082783ad01b2714bc'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:59:18.177727 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-69f7d69a-e726-418c-b176-a6428e1d714b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5f6cb25e6a164de082783ad01b2714bc'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:59:18.178329 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-69f7d69a-e726-418c-b176-a6428e1d714b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5f6cb25e6a164de082783ad01b2714bc'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:59:18.178970 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-69f7d69a-e726-418c-b176-a6428e1d714b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5f6cb25e6a164de082783ad01b2714bc'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:59:18.196364 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1404/2832] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Jul 27 10:59:18 2022] DELETE /identity/v3/users/5f6cb25e6a164de082783ad01b2714bc => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:59:18.200814 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c864bfa5-1e88-45ea-844f-bea7ea873f1b None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:59:18.201305 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c864bfa5-1e88-45ea-844f-bea7ea873f1b None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:59:18.201426 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c864bfa5-1e88-45ea-844f-bea7ea873f1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:59:18.219664 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-c864bfa5-1e88-45ea-844f-bea7ea873f1b None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:59:18.220858 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-c864bfa5-1e88-45ea-844f-bea7ea873f1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:59:18.267497 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:DELETE=6,keystone:SELECT=91,keystone:INSERT=3 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:59:18.281144 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1429/2833] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:59:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:59:18.290970 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f7d69a-e726-418c-b176-a6428e1d714b admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:59:18.292353 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-69f7d69a-e726-418c-b176-a6428e1d714b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:59:18.306073 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:59:18.350019 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f7d69a-e726-418c-b176-a6428e1d714b admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:59:18.353554 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09b15e9-cc09-4fab-bdbf-a93430c163a9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7922d0a1a7c4e808d0b2a80dca0da40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:59:18.354158 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09b15e9-cc09-4fab-bdbf-a93430c163a9 service neutron] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:59:18.354437 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09b15e9-cc09-4fab-bdbf-a93430c163a9 service neutron] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:59:18.354767 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09b15e9-cc09-4fab-bdbf-a93430c163a9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:59:18.364081 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09b15e9-cc09-4fab-bdbf-a93430c163a9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:59:18.365534 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09b15e9-cc09-4fab-bdbf-a93430c163a9 service neutron] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:59:18.374979 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1405/2834] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:59:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 10:59:18.657533 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c864bfa5-1e88-45ea-844f-bea7ea873f1b None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:59:18.659043 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-c864bfa5-1e88-45ea-844f-bea7ea873f1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:59:18.671962 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c864bfa5-1e88-45ea-844f-bea7ea873f1b None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:59:18.677378 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b0070e-7c34-4f95-9096-cab05c09a46e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:59:18.678502 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b0070e-7c34-4f95-9096-cab05c09a46e admin admin] REQUEST_METHOD: `DELETE` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:59:18.678892 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b0070e-7c34-4f95-9096-cab05c09a46e admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:59:18.679338 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b0070e-7c34-4f95-9096-cab05c09a46e admin admin] PATH_INFO: `/v3/projects/49a315768bd94707adeb4e9f0eb12a8c` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:59:18.680692 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b0070e-7c34-4f95-9096-cab05c09a46e admin admin] RBAC: Authorizing `identity:delete_project(project_id=49a315768bd94707adeb4e9f0eb12a8c)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:59:18.682496 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b0070e-7c34-4f95-9096-cab05c09a46e admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:59:18.729175 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-18b0070e-7c34-4f95-9096-cab05c09a46e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '49a315768bd94707adeb4e9f0eb12a8c', 'request_id': 'req-18b0070e-7c34-4f95-9096-cab05c09a46e'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:59:18.735632 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-18b0070e-7c34-4f95-9096-cab05c09a46e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '49a315768bd94707adeb4e9f0eb12a8c', 'request_id': 'req-18b0070e-7c34-4f95-9096-cab05c09a46e'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:59:18.742852 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-18b0070e-7c34-4f95-9096-cab05c09a46e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '49a315768bd94707adeb4e9f0eb12a8c', 'request_id': 'req-18b0070e-7c34-4f95-9096-cab05c09a46e'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:59:18.753422 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-18b0070e-7c34-4f95-9096-cab05c09a46e admin admin] The token cache is being invalidate because project 49a315768bd94707adeb4e9f0eb12a8c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86827) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 10:59:18.754238 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-18b0070e-7c34-4f95-9096-cab05c09a46e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:59:18.754942 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-18b0070e-7c34-4f95-9096-cab05c09a46e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 10:59:18.756541 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1430/2835] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Jul 27 10:59:18 2022] DELETE /identity/v3/projects/49a315768bd94707adeb4e9f0eb12a8c => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:59:18.761492 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdfeb17-774b-422c-bfb6-3155fb9d4d95 None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:59:18.761786 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdfeb17-774b-422c-bfb6-3155fb9d4d95 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:59:18.762064 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdfeb17-774b-422c-bfb6-3155fb9d4d95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:59:18.781928 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-acdfeb17-774b-422c-bfb6-3155fb9d4d95 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:59:18.783178 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-acdfeb17-774b-422c-bfb6-3155fb9d4d95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:59:18.841334 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1406/2836] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:59:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:59:18.849240 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c8446e-f829-43f6-bffc-6c1ae5c9e354 None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:59:18.849542 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c8446e-f829-43f6-bffc-6c1ae5c9e354 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:59:18.849823 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c8446e-f829-43f6-bffc-6c1ae5c9e354 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:59:18.873989 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-e5c8446e-f829-43f6-bffc-6c1ae5c9e354 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:59:18.875156 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-e5c8446e-f829-43f6-bffc-6c1ae5c9e354 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:59:18.883858 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1431/2837] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:59:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:59:18.888096 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdfeb17-774b-422c-bfb6-3155fb9d4d95 None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:59:18.897705 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdfeb17-774b-422c-bfb6-3155fb9d4d95 None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:59:18.902236 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbfe113-f86f-4aad-99c8-fb8a115a4529 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:59:18.902882 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbfe113-f86f-4aad-99c8-fb8a115a4529 admin admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:59:18.903161 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbfe113-f86f-4aad-99c8-fb8a115a4529 admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:59:18.903427 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbfe113-f86f-4aad-99c8-fb8a115a4529 admin admin] PATH_INFO: `/v3/domains` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:59:18.904113 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbfe113-f86f-4aad-99c8-fb8a115a4529 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86828) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 10:59:18.904413 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbfe113-f86f-4aad-99c8-fb8a115a4529 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:59:18.905982 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbfe113-f86f-4aad-99c8-fb8a115a4529 admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:59:18.917213 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1407/2838] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Jul 27 10:59:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 10:59:18.921295 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c8446e-f829-43f6-bffc-6c1ae5c9e354 None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:59:18.931092 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c8446e-f829-43f6-bffc-6c1ae5c9e354 None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:59:18.935740 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707b5dd5-c9d5-49c2-af0e-27d768fda624 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:59:18.936342 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707b5dd5-c9d5-49c2-af0e-27d768fda624 admin admin] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:59:18.936627 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707b5dd5-c9d5-49c2-af0e-27d768fda624 admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:59:18.936902 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707b5dd5-c9d5-49c2-af0e-27d768fda624 admin admin] PATH_INFO: `/v3/projects` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:59:18.937574 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707b5dd5-c9d5-49c2-af0e-27d768fda624 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:59:18.939505 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707b5dd5-c9d5-49c2-af0e-27d768fda624 admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:59:18.967732 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1432/2839] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Jul 27 10:59:18 2022] POST /identity/v3/projects => generated 371 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 10:59:18.971884 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbfe113-f86f-4aad-99c8-fb8a115a4529 admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:59:18.981758 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbfe113-f86f-4aad-99c8-fb8a115a4529 admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:59:19.036231 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec52554-1fa9-4bc5-9958-36059aca7bfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:59:19.036837 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec52554-1fa9-4bc5-9958-36059aca7bfe admin admin] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:59:19.037117 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec52554-1fa9-4bc5-9958-36059aca7bfe admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:59:19.037687 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec52554-1fa9-4bc5-9958-36059aca7bfe admin admin] PATH_INFO: `/v3/users` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:59:19.038033 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec52554-1fa9-4bc5-9958-36059aca7bfe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:59:19.039961 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec52554-1fa9-4bc5-9958-36059aca7bfe admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:59:19.060522 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1408/2840] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Jul 27 10:59:18 2022] POST /identity/v3/users => generated 286 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 10:59:19.064652 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707b5dd5-c9d5-49c2-af0e-27d768fda624 admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:59:19.075443 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707b5dd5-c9d5-49c2-af0e-27d768fda624 admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:59:19.079987 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bbcf2b-03c0-4669-94e8-cd75e4bdcb36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:59:19.080590 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bbcf2b-03c0-4669-94e8-cd75e4bdcb36 admin admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:59:19.080872 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bbcf2b-03c0-4669-94e8-cd75e4bdcb36 admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:59:19.081147 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bbcf2b-03c0-4669-94e8-cd75e4bdcb36 admin admin] PATH_INFO: `/v3/roles` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:59:19.081836 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79bbcf2b-03c0-4669-94e8-cd75e4bdcb36 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:59:19.083456 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79bbcf2b-03c0-4669-94e8-cd75e4bdcb36 admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:59:19.093160 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1433/2841] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 10:59:19 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 10:59:19.096853 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec52554-1fa9-4bc5-9958-36059aca7bfe admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:59:19.106588 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec52554-1fa9-4bc5-9958-36059aca7bfe admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:59:19.111150 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80522711-8a40-4796-8321-cd95e01e8f20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:59:19.111746 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80522711-8a40-4796-8321-cd95e01e8f20 admin admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:59:19.112030 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80522711-8a40-4796-8321-cd95e01e8f20 admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:59:19.112303 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80522711-8a40-4796-8321-cd95e01e8f20 admin admin] PATH_INFO: `/v3/roles` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:59:19.112978 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80522711-8a40-4796-8321-cd95e01e8f20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:59:19.115089 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80522711-8a40-4796-8321-cd95e01e8f20 admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:59:19.126644 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1409/2842] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 10:59:19 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 10:59:19.131130 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bbcf2b-03c0-4669-94e8-cd75e4bdcb36 admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:59:19.140825 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bbcf2b-03c0-4669-94e8-cd75e4bdcb36 admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:59:19.145464 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec946407-c6c7-4cf3-b379-a3fbb6971335 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:59:19.146109 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec946407-c6c7-4cf3-b379-a3fbb6971335 admin admin] REQUEST_METHOD: `PUT` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:59:19.146392 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec946407-c6c7-4cf3-b379-a3fbb6971335 admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:59:19.146702 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec946407-c6c7-4cf3-b379-a3fbb6971335 admin admin] PATH_INFO: `/v3/projects/dcc755e972c1451e98b1f2ce974e4ff0/users/f2cdf37458874483a778c0e6c55d9232/roles/4c9fc0b3baed441da2126be41cc6b04a` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:59:19.161058 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec946407-c6c7-4cf3-b379-a3fbb6971335 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dcc755e972c1451e98b1f2ce974e4ff0, user_id=f2cdf37458874483a778c0e6c55d9232, role_id=4c9fc0b3baed441da2126be41cc6b04a)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:59:19.163927 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec946407-c6c7-4cf3-b379-a3fbb6971335 admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:59:19.177814 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1434/2843] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 10:59:19 2022] PUT /identity/v3/projects/dcc755e972c1451e98b1f2ce974e4ff0/users/f2cdf37458874483a778c0e6c55d9232/roles/4c9fc0b3baed441da2126be41cc6b04a => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:59:19.182850 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edb490e-8ab1-46e5-a088-95f67ad79cbf None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:59:19.183216 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edb490e-8ab1-46e5-a088-95f67ad79cbf None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:59:19.183521 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edb490e-8ab1-46e5-a088-95f67ad79cbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:59:19.202183 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-9edb490e-8ab1-46e5-a088-95f67ad79cbf None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:59:19.203425 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-9edb490e-8ab1-46e5-a088-95f67ad79cbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:59:19.216220 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:59:19.263887 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1410/2844] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:59:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:59:19.274468 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec946407-c6c7-4cf3-b379-a3fbb6971335 admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:59:19.276175 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-ec946407-c6c7-4cf3-b379-a3fbb6971335 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:59:19.332609 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec946407-c6c7-4cf3-b379-a3fbb6971335 admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:59:19.336270 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04311a51-1028-4a16-ba77-dd969a063738 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7922d0a1a7c4e808d0b2a80dca0da40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:59:19.336887 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04311a51-1028-4a16-ba77-dd969a063738 service neutron] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:59:19.337168 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04311a51-1028-4a16-ba77-dd969a063738 service neutron] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:59:19.337441 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04311a51-1028-4a16-ba77-dd969a063738 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:59:19.346408 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04311a51-1028-4a16-ba77-dd969a063738 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:59:19.347896 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04311a51-1028-4a16-ba77-dd969a063738 service neutron] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:59:19.357824 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1435/2845] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:59:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 10:59:23.356353 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5cf7a4-d7a2-4d6e-a304-1bc3af678834 None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:59:23.356876 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5cf7a4-d7a2-4d6e-a304-1bc3af678834 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:59:23.356876 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5cf7a4-d7a2-4d6e-a304-1bc3af678834 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:59:23.386923 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-2e5cf7a4-d7a2-4d6e-a304-1bc3af678834 None None] MFA Rules not processed for user `f2cdf37458874483a778c0e6c55d9232`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:59:23.388066 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-2e5cf7a4-d7a2-4d6e-a304-1bc3af678834 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:59:23.461298 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1411/2846] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:59:23 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 10:59:23.465684 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04311a51-1028-4a16-ba77-dd969a063738 service neutron] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:59:23.476371 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04311a51-1028-4a16-ba77-dd969a063738 service neutron] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:59:23.481317 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b2c3c1-3150-4b6d-9417-74de38a4951b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:59:23.482039 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b2c3c1-3150-4b6d-9417-74de38a4951b admin admin] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:59:23.482330 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b2c3c1-3150-4b6d-9417-74de38a4951b admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:59:23.482638 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b2c3c1-3150-4b6d-9417-74de38a4951b admin admin] PATH_INFO: `/v3/projects` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:59:23.483339 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b2c3c1-3150-4b6d-9417-74de38a4951b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:59:23.485257 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b2c3c1-3150-4b6d-9417-74de38a4951b admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:59:23.502774 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:59:23.518077 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1436/2847] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Jul 27 10:59:23 2022] POST /identity/v3/projects => generated 371 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 10:59:23.522224 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5cf7a4-d7a2-4d6e-a304-1bc3af678834 None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:59:23.533453 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5cf7a4-d7a2-4d6e-a304-1bc3af678834 None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:59:23.592146 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeba4763-8205-406c-b6f2-64c89090f005 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:59:23.592777 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeba4763-8205-406c-b6f2-64c89090f005 admin admin] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:59:23.593143 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeba4763-8205-406c-b6f2-64c89090f005 admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:59:23.593425 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeba4763-8205-406c-b6f2-64c89090f005 admin admin] PATH_INFO: `/v3/users` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:59:23.594106 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeba4763-8205-406c-b6f2-64c89090f005 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:59:23.596051 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeba4763-8205-406c-b6f2-64c89090f005 admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:59:23.618632 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1412/2848] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Jul 27 10:59:23 2022] POST /identity/v3/users => generated 292 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 10:59:23.623124 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b2c3c1-3150-4b6d-9417-74de38a4951b admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:59:23.634716 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b2c3c1-3150-4b6d-9417-74de38a4951b admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:59:23.639496 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96c900a-5a59-43e3-a984-bbb7d9557a2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:59:23.640055 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96c900a-5a59-43e3-a984-bbb7d9557a2f admin admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:59:23.640340 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96c900a-5a59-43e3-a984-bbb7d9557a2f admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:59:23.640756 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96c900a-5a59-43e3-a984-bbb7d9557a2f admin admin] PATH_INFO: `/v3/roles` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:59:23.641669 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96c900a-5a59-43e3-a984-bbb7d9557a2f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:59:23.643340 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96c900a-5a59-43e3-a984-bbb7d9557a2f admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:59:23.654856 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1437/2849] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 10:59:23 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 10:59:23.659055 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeba4763-8205-406c-b6f2-64c89090f005 admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:59:23.669840 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeba4763-8205-406c-b6f2-64c89090f005 admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:59:23.674564 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe65e40-605b-45be-9a18-b026eb54a5b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:59:23.675272 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe65e40-605b-45be-9a18-b026eb54a5b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:59:23.676080 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe65e40-605b-45be-9a18-b026eb54a5b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:59:23.676355 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe65e40-605b-45be-9a18-b026eb54a5b3 admin admin] PATH_INFO: `/v3/projects/94b763d4411f4205b7a176b6db443c9a/users/cdeb6f91dd174468a7e71ac7a17c9a95/roles/21156560eb6b436bb567a40b4976f3f4` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:59:23.693425 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe65e40-605b-45be-9a18-b026eb54a5b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=94b763d4411f4205b7a176b6db443c9a, user_id=cdeb6f91dd174468a7e71ac7a17c9a95, role_id=21156560eb6b436bb567a40b4976f3f4)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:59:23.696536 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe65e40-605b-45be-9a18-b026eb54a5b3 admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:59:23.711154 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1413/2850] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 10:59:23 2022] PUT /identity/v3/projects/94b763d4411f4205b7a176b6db443c9a/users/cdeb6f91dd174468a7e71ac7a17c9a95/roles/21156560eb6b436bb567a40b4976f3f4 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:59:23.715533 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96c900a-5a59-43e3-a984-bbb7d9557a2f admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:59:23.726455 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96c900a-5a59-43e3-a984-bbb7d9557a2f admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:59:23.789580 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171f28c7-9a82-45e0-adde-d5464b3ab0a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:59:23.790290 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171f28c7-9a82-45e0-adde-d5464b3ab0a6 admin admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:59:23.790574 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171f28c7-9a82-45e0-adde-d5464b3ab0a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:59:23.790889 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171f28c7-9a82-45e0-adde-d5464b3ab0a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:59:23.791581 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171f28c7-9a82-45e0-adde-d5464b3ab0a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:59:23.793178 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171f28c7-9a82-45e0-adde-d5464b3ab0a6 admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:59:23.804827 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1438/2851] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 10:59:23 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 10:59:23.809047 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe65e40-605b-45be-9a18-b026eb54a5b3 admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:59:23.819652 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe65e40-605b-45be-9a18-b026eb54a5b3 admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:59:23.824214 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b5c25e-2aaa-4f55-a51b-f6689c2a5bcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:59:23.824874 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b5c25e-2aaa-4f55-a51b-f6689c2a5bcc admin admin] REQUEST_METHOD: `PUT` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:59:23.825154 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b5c25e-2aaa-4f55-a51b-f6689c2a5bcc admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:59:23.825428 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b5c25e-2aaa-4f55-a51b-f6689c2a5bcc admin admin] PATH_INFO: `/v3/projects/94b763d4411f4205b7a176b6db443c9a/users/cdeb6f91dd174468a7e71ac7a17c9a95/roles/21156560eb6b436bb567a40b4976f3f4` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:59:23.827376 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b5c25e-2aaa-4f55-a51b-f6689c2a5bcc admin admin] RBAC: Authorizing `identity:create_grant(project_id=94b763d4411f4205b7a176b6db443c9a, user_id=cdeb6f91dd174468a7e71ac7a17c9a95, role_id=21156560eb6b436bb567a40b4976f3f4)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:59:23.830034 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b5c25e-2aaa-4f55-a51b-f6689c2a5bcc admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:59:23.840179 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1414/2852] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 10:59:23 2022] PUT /identity/v3/projects/94b763d4411f4205b7a176b6db443c9a/users/cdeb6f91dd174468a7e71ac7a17c9a95/roles/21156560eb6b436bb567a40b4976f3f4 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 10:59:28.326130 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce3f268-66ec-4a01-a96f-d48a310e6013 None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:59:28.326727 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce3f268-66ec-4a01-a96f-d48a310e6013 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:59:28.326898 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce3f268-66ec-4a01-a96f-d48a310e6013 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:59:28.373795 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-1ce3f268-66ec-4a01-a96f-d48a310e6013 None None] MFA Rules not processed for user `cdeb6f91dd174468a7e71ac7a17c9a95`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:59:28.375213 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-1ce3f268-66ec-4a01-a96f-d48a310e6013 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:59:28.496284 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1439/2853] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:59:28 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:59:28.515818 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b5c25e-2aaa-4f55-a51b-f6689c2a5bcc admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:59:28.519994 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-31b5c25e-2aaa-4f55-a51b-f6689c2a5bcc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:59:28.530783 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:59:28.612320 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b5c25e-2aaa-4f55-a51b-f6689c2a5bcc admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:59:28.620895 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a07576-4e85-467d-a0d1-fd4c5833c3f3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e059d8b6c28c4fcba925d05d7c98e4ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:59:28.621695 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a07576-4e85-467d-a0d1-fd4c5833c3f3 service nova] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:59:28.622082 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a07576-4e85-467d-a0d1-fd4c5833c3f3 service nova] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:59:28.622462 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a07576-4e85-467d-a0d1-fd4c5833c3f3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:59:28.684738 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a07576-4e85-467d-a0d1-fd4c5833c3f3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:59:28.686629 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a07576-4e85-467d-a0d1-fd4c5833c3f3 service nova] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:59:28.697132 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1415/2854] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:59:28 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 10:59:29.664054 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c72993-7b2b-4a17-ae53-bfc2ccf714fc None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:59:29.664589 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c72993-7b2b-4a17-ae53-bfc2ccf714fc None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:59:29.664589 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c72993-7b2b-4a17-ae53-bfc2ccf714fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:59:29.683596 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-51c72993-7b2b-4a17-ae53-bfc2ccf714fc None None] MFA Rules not processed for user `f2cdf37458874483a778c0e6c55d9232`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 10:59:29.684796 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-51c72993-7b2b-4a17-ae53-bfc2ccf714fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:59:29.692123 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1440/2855] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 10:59:29 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 10:59:38.692665 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=55 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:59:39.677964 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:INSERT=1,keystone:SELECT=73 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:59:46.668508 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a07576-4e85-467d-a0d1-fd4c5833c3f3 service nova] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:59:46.669960 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-f1a07576-4e85-467d-a0d1-fd4c5833c3f3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:59:46.740940 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a07576-4e85-467d-a0d1-fd4c5833c3f3 service nova] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:59:46.746384 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3caa27-ca19-4846-86c1-1ffc03cafc97 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16cc6179e9a04b6398fbacb40ad1c555', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:59:46.747050 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3caa27-ca19-4846-86c1-1ffc03cafc97 service placement] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:59:46.747332 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3caa27-ca19-4846-86c1-1ffc03cafc97 service placement] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:59:46.747715 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3caa27-ca19-4846-86c1-1ffc03cafc97 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:59:46.749453 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-0e3caa27-ca19-4846-86c1-1ffc03cafc97 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:59:46.760977 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3caa27-ca19-4846-86c1-1ffc03cafc97 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:59:46.762551 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3caa27-ca19-4846-86c1-1ffc03cafc97 service placement] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:59:46.772361 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1416/2856] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:59:46 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 10:59:48.388599 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c72993-7b2b-4a17-ae53-bfc2ccf714fc None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:59:48.448531 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c72993-7b2b-4a17-ae53-bfc2ccf714fc None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:59:48.452151 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08947fc-b046-4586-b5a4-b7da83aed482 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7922d0a1a7c4e808d0b2a80dca0da40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:59:48.452770 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08947fc-b046-4586-b5a4-b7da83aed482 service neutron] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:59:48.453050 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08947fc-b046-4586-b5a4-b7da83aed482 service neutron] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:59:48.453319 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08947fc-b046-4586-b5a4-b7da83aed482 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:59:48.455103 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-f08947fc-b046-4586-b5a4-b7da83aed482 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:59:48.465124 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08947fc-b046-4586-b5a4-b7da83aed482 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:59:48.466519 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08947fc-b046-4586-b5a4-b7da83aed482 service neutron] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:59:48.474710 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1441/2857] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:59:48 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 10:59:49.110950 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3caa27-ca19-4846-86c1-1ffc03cafc97 service placement] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:59:49.112441 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-0e3caa27-ca19-4846-86c1-1ffc03cafc97 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:59:49.141932 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3caa27-ca19-4846-86c1-1ffc03cafc97 service placement] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:59:49.152382 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b78dfda-8ced-4513-8b5e-b57f4fdd3653 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e059d8b6c28c4fcba925d05d7c98e4ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:59:49.153023 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b78dfda-8ced-4513-8b5e-b57f4fdd3653 service nova] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:59:49.153365 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b78dfda-8ced-4513-8b5e-b57f4fdd3653 service nova] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:59:49.153648 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b78dfda-8ced-4513-8b5e-b57f4fdd3653 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:59:49.155517 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-8b78dfda-8ced-4513-8b5e-b57f4fdd3653 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:59:49.170810 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b78dfda-8ced-4513-8b5e-b57f4fdd3653 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:59:49.172438 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b78dfda-8ced-4513-8b5e-b57f4fdd3653 service nova] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:59:49.184705 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1417/2858] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:59:49 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 75 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 10:59:58.471460 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=26 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 10:59:58.679297 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08947fc-b046-4586-b5a4-b7da83aed482 service neutron] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 10:59:58.693800 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08947fc-b046-4586-b5a4-b7da83aed482 service neutron] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 10:59:58.699639 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19cfd72-bc50-4052-9e72-a77bbd1ae2b1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e059d8b6c28c4fcba925d05d7c98e4ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 10:59:58.700261 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19cfd72-bc50-4052-9e72-a77bbd1ae2b1 service nova] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 10:59:58.700548 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19cfd72-bc50-4052-9e72-a77bbd1ae2b1 service nova] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 10:59:58.700930 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19cfd72-bc50-4052-9e72-a77bbd1ae2b1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 10:59:58.702723 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-f19cfd72-bc50-4052-9e72-a77bbd1ae2b1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 10:59:58.715804 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19cfd72-bc50-4052-9e72-a77bbd1ae2b1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 10:59:58.717612 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19cfd72-bc50-4052-9e72-a77bbd1ae2b1 service nova] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 10:59:58.729062 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1442/2859] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 10:59:58 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 10:59:59.180647 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=38 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 11:00:00.873671 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b78dfda-8ced-4513-8b5e-b57f4fdd3653 service nova] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 11:00:00.875250 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-8b78dfda-8ced-4513-8b5e-b57f4fdd3653 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 11:00:00.887004 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b78dfda-8ced-4513-8b5e-b57f4fdd3653 service nova] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 11:00:00.890545 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4879e3ca-67d7-415b-921c-7a64b3834fe3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7922d0a1a7c4e808d0b2a80dca0da40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 11:00:00.891200 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4879e3ca-67d7-415b-921c-7a64b3834fe3 service neutron] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 11:00:00.891479 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4879e3ca-67d7-415b-921c-7a64b3834fe3 service neutron] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 11:00:00.891751 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4879e3ca-67d7-415b-921c-7a64b3834fe3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 11:00:00.901043 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4879e3ca-67d7-415b-921c-7a64b3834fe3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 11:00:00.902452 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4879e3ca-67d7-415b-921c-7a64b3834fe3 service neutron] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 11:00:00.911689 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1418/2860] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 11:00:00 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 11:00:02.728818 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19cfd72-bc50-4052-9e72-a77bbd1ae2b1 service nova] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 11:00:02.743847 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19cfd72-bc50-4052-9e72-a77bbd1ae2b1 service nova] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 11:00:02.751733 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114faf58-595a-4ba4-b18c-f8e1a3ac9a98 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e059d8b6c28c4fcba925d05d7c98e4ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 11:00:02.752360 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114faf58-595a-4ba4-b18c-f8e1a3ac9a98 service nova] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 11:00:02.752641 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114faf58-595a-4ba4-b18c-f8e1a3ac9a98 service nova] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 11:00:02.752913 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114faf58-595a-4ba4-b18c-f8e1a3ac9a98 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 11:00:02.756097 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-114faf58-595a-4ba4-b18c-f8e1a3ac9a98 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 11:00:02.769372 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114faf58-595a-4ba4-b18c-f8e1a3ac9a98 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 11:00:02.771007 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114faf58-595a-4ba4-b18c-f8e1a3ac9a98 service nova] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 11:00:02.781769 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1443/2861] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 11:00:02 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 11:00:10.907570 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=6 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 11:00:12.777370 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=12 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 11:01:07.806029 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4879e3ca-67d7-415b-921c-7a64b3834fe3 service neutron] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 11:01:07.807469 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-4879e3ca-67d7-415b-921c-7a64b3834fe3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 11:01:07.821964 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4879e3ca-67d7-415b-921c-7a64b3834fe3 service neutron] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 11:01:07.826948 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95ee3f8-675a-4c73-8a51-32c20abb4e1b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16cc6179e9a04b6398fbacb40ad1c555', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 11:01:07.827561 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95ee3f8-675a-4c73-8a51-32c20abb4e1b service placement] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 11:01:07.827848 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95ee3f8-675a-4c73-8a51-32c20abb4e1b service placement] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 11:01:07.828156 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95ee3f8-675a-4c73-8a51-32c20abb4e1b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 11:01:07.829822 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-b95ee3f8-675a-4c73-8a51-32c20abb4e1b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 11:01:07.841606 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95ee3f8-675a-4c73-8a51-32c20abb4e1b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 11:01:07.843218 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95ee3f8-675a-4c73-8a51-32c20abb4e1b service placement] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 11:01:07.853647 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1419/2862] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 11:01:07 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 11:01:17.848791 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=6 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 11:02:07.745612 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114faf58-595a-4ba4-b18c-f8e1a3ac9a98 service nova] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 11:02:07.755230 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114faf58-595a-4ba4-b18c-f8e1a3ac9a98 service nova] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 11:02:07.759444 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51eeb4f8-e11c-42e1-b5ac-872b5c48a459 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7922d0a1a7c4e808d0b2a80dca0da40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 11:02:07.760205 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51eeb4f8-e11c-42e1-b5ac-872b5c48a459 service neutron] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 11:02:07.760525 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51eeb4f8-e11c-42e1-b5ac-872b5c48a459 service neutron] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 11:02:07.760890 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51eeb4f8-e11c-42e1-b5ac-872b5c48a459 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 11:02:07.762832 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-51eeb4f8-e11c-42e1-b5ac-872b5c48a459 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 11:02:07.773028 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51eeb4f8-e11c-42e1-b5ac-872b5c48a459 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 11:02:07.774438 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51eeb4f8-e11c-42e1-b5ac-872b5c48a459 service neutron] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 11:02:07.783034 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1444/2863] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 11:02:07 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 11:02:08.890628 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95ee3f8-675a-4c73-8a51-32c20abb4e1b service placement] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 11:02:08.900058 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95ee3f8-675a-4c73-8a51-32c20abb4e1b service placement] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 11:02:08.903661 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77e7bcd-4e7d-488a-95fd-f49e29f51369 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7922d0a1a7c4e808d0b2a80dca0da40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 11:02:08.904763 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77e7bcd-4e7d-488a-95fd-f49e29f51369 service neutron] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 11:02:08.905045 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77e7bcd-4e7d-488a-95fd-f49e29f51369 service neutron] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 11:02:08.905371 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77e7bcd-4e7d-488a-95fd-f49e29f51369 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 11:02:08.907123 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-b77e7bcd-4e7d-488a-95fd-f49e29f51369 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 11:02:08.916871 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77e7bcd-4e7d-488a-95fd-f49e29f51369 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 11:02:08.918298 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77e7bcd-4e7d-488a-95fd-f49e29f51369 service neutron] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 11:02:08.927578 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1420/2864] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 11:02:08 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 11:02:10.033375 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51eeb4f8-e11c-42e1-b5ac-872b5c48a459 service neutron] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 11:02:10.042357 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51eeb4f8-e11c-42e1-b5ac-872b5c48a459 service neutron] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 11:02:10.046506 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23230f6-5892-40fe-982b-22b156d6b6e7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7922d0a1a7c4e808d0b2a80dca0da40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 11:02:10.047139 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23230f6-5892-40fe-982b-22b156d6b6e7 service neutron] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 11:02:10.047426 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23230f6-5892-40fe-982b-22b156d6b6e7 service neutron] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 11:02:10.047705 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23230f6-5892-40fe-982b-22b156d6b6e7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 11:02:10.049467 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-d23230f6-5892-40fe-982b-22b156d6b6e7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 11:02:10.065306 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23230f6-5892-40fe-982b-22b156d6b6e7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 11:02:10.066681 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23230f6-5892-40fe-982b-22b156d6b6e7 service neutron] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 11:02:10.078180 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1445/2865] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 11:02:10 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 11:02:18.923417 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=6 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 11:02:20.073732 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=12 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 11:03:42.063244 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77e7bcd-4e7d-488a-95fd-f49e29f51369 service neutron] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 11:03:42.072536 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77e7bcd-4e7d-488a-95fd-f49e29f51369 service neutron] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 11:03:42.076362 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88314ae2-13c8-49a2-9fbb-476f7c841c46 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7922d0a1a7c4e808d0b2a80dca0da40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 11:03:42.076815 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88314ae2-13c8-49a2-9fbb-476f7c841c46 service neutron] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 11:03:42.077097 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88314ae2-13c8-49a2-9fbb-476f7c841c46 service neutron] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 11:03:42.077371 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88314ae2-13c8-49a2-9fbb-476f7c841c46 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 11:03:42.079248 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-88314ae2-13c8-49a2-9fbb-476f7c841c46 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 11:03:42.097361 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88314ae2-13c8-49a2-9fbb-476f7c841c46 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 11:03:42.098860 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88314ae2-13c8-49a2-9fbb-476f7c841c46 service neutron] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 11:03:42.110572 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1421/2866] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 11:03:42 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 11:03:43.287437 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23230f6-5892-40fe-982b-22b156d6b6e7 service neutron] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 11:03:43.296039 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23230f6-5892-40fe-982b-22b156d6b6e7 service neutron] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 11:03:43.300187 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826bae2f-e582-4d91-8ac7-2cd5a4f0281d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7922d0a1a7c4e808d0b2a80dca0da40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 11:03:43.300798 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826bae2f-e582-4d91-8ac7-2cd5a4f0281d service neutron] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 11:03:43.301081 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826bae2f-e582-4d91-8ac7-2cd5a4f0281d service neutron] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 11:03:43.301354 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826bae2f-e582-4d91-8ac7-2cd5a4f0281d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 11:03:43.303104 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-826bae2f-e582-4d91-8ac7-2cd5a4f0281d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 11:03:43.316953 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826bae2f-e582-4d91-8ac7-2cd5a4f0281d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 11:03:43.318365 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826bae2f-e582-4d91-8ac7-2cd5a4f0281d service neutron] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 11:03:43.329265 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1446/2867] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 11:03:43 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 11:03:49.841307 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88314ae2-13c8-49a2-9fbb-476f7c841c46 service neutron] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 11:03:49.852710 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88314ae2-13c8-49a2-9fbb-476f7c841c46 service neutron] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 11:03:49.858310 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302f5378-36cf-4fec-b578-31ffc6730456 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16cc6179e9a04b6398fbacb40ad1c555', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 11:03:49.859207 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302f5378-36cf-4fec-b578-31ffc6730456 service placement] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 11:03:49.859493 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302f5378-36cf-4fec-b578-31ffc6730456 service placement] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 11:03:49.859770 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302f5378-36cf-4fec-b578-31ffc6730456 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 11:03:49.861775 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-302f5378-36cf-4fec-b578-31ffc6730456 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 11:03:49.874669 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302f5378-36cf-4fec-b578-31ffc6730456 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 11:03:49.877102 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302f5378-36cf-4fec-b578-31ffc6730456 service placement] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 11:03:49.888712 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1422/2868] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 11:03:49 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 11:03:53.030926 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826bae2f-e582-4d91-8ac7-2cd5a4f0281d service neutron] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 11:03:53.094648 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826bae2f-e582-4d91-8ac7-2cd5a4f0281d service neutron] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 11:03:53.099155 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad58856e-ee25-4fbb-b2e8-f41194ab6335 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 11:03:53.099790 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad58856e-ee25-4fbb-b2e8-f41194ab6335 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 11:03:53.100076 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad58856e-ee25-4fbb-b2e8-f41194ab6335 admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 11:03:53.100354 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad58856e-ee25-4fbb-b2e8-f41194ab6335 admin admin] PATH_INFO: `/v3/users/f2cdf37458874483a778c0e6c55d9232` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 11:03:53.101518 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad58856e-ee25-4fbb-b2e8-f41194ab6335 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f2cdf37458874483a778c0e6c55d9232)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 11:03:53.103460 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad58856e-ee25-4fbb-b2e8-f41194ab6335 admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 11:03:53.149004 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-ad58856e-ee25-4fbb-b2e8-f41194ab6335 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f2cdf37458874483a778c0e6c55d9232'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 11:03:53.156189 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-ad58856e-ee25-4fbb-b2e8-f41194ab6335 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f2cdf37458874483a778c0e6c55d9232'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 11:03:53.164280 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-ad58856e-ee25-4fbb-b2e8-f41194ab6335 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f2cdf37458874483a778c0e6c55d9232'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 11:03:53.165039 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-ad58856e-ee25-4fbb-b2e8-f41194ab6335 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f2cdf37458874483a778c0e6c55d9232'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 11:03:53.165690 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-ad58856e-ee25-4fbb-b2e8-f41194ab6335 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f2cdf37458874483a778c0e6c55d9232'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 11:03:53.183292 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1447/2869] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Jul 27 11:03:53 2022] DELETE /identity/v3/users/f2cdf37458874483a778c0e6c55d9232 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 11:03:53.349943 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302f5378-36cf-4fec-b578-31ffc6730456 service placement] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 11:03:53.361421 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302f5378-36cf-4fec-b578-31ffc6730456 service placement] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 11:03:53.422983 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3beb58-0bbf-4254-b937-20ae45c35675 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 11:03:53.424243 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3beb58-0bbf-4254-b937-20ae45c35675 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 11:03:53.424576 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3beb58-0bbf-4254-b937-20ae45c35675 admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 11:03:53.424856 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3beb58-0bbf-4254-b937-20ae45c35675 admin admin] PATH_INFO: `/v3/projects/dcc755e972c1451e98b1f2ce974e4ff0` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 11:03:53.426033 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3beb58-0bbf-4254-b937-20ae45c35675 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dcc755e972c1451e98b1f2ce974e4ff0)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 11:03:53.427978 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3beb58-0bbf-4254-b937-20ae45c35675 admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 11:03:53.473681 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-2f3beb58-0bbf-4254-b937-20ae45c35675 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dcc755e972c1451e98b1f2ce974e4ff0', 'request_id': 'req-2f3beb58-0bbf-4254-b937-20ae45c35675'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 11:03:53.479628 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-2f3beb58-0bbf-4254-b937-20ae45c35675 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dcc755e972c1451e98b1f2ce974e4ff0', 'request_id': 'req-2f3beb58-0bbf-4254-b937-20ae45c35675'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 11:03:53.487388 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-2f3beb58-0bbf-4254-b937-20ae45c35675 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dcc755e972c1451e98b1f2ce974e4ff0', 'request_id': 'req-2f3beb58-0bbf-4254-b937-20ae45c35675'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 11:03:53.497778 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-2f3beb58-0bbf-4254-b937-20ae45c35675 admin admin] The token cache is being invalidate because project dcc755e972c1451e98b1f2ce974e4ff0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86828) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 11:03:53.498071 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-2f3beb58-0bbf-4254-b937-20ae45c35675 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 11:03:53.498717 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-2f3beb58-0bbf-4254-b937-20ae45c35675 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 11:03:53.500194 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1423/2870] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Jul 27 11:03:53 2022] DELETE /identity/v3/projects/dcc755e972c1451e98b1f2ce974e4ff0 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 11:03:53.504211 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad58856e-ee25-4fbb-b2e8-f41194ab6335 admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 11:03:53.505641 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-ad58856e-ee25-4fbb-b2e8-f41194ab6335 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 11:03:53.518507 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad58856e-ee25-4fbb-b2e8-f41194ab6335 admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 11:03:53.574924 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78351ab-f65d-4b15-8ab9-36ba1095648a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 11:03:53.575565 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78351ab-f65d-4b15-8ab9-36ba1095648a admin admin] REQUEST_METHOD: `DELETE` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 11:03:53.575848 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78351ab-f65d-4b15-8ab9-36ba1095648a admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 11:03:53.576120 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78351ab-f65d-4b15-8ab9-36ba1095648a admin admin] PATH_INFO: `/v3/users/cdeb6f91dd174468a7e71ac7a17c9a95` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 11:03:53.577282 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78351ab-f65d-4b15-8ab9-36ba1095648a admin admin] RBAC: Authorizing `identity:delete_user(user_id=cdeb6f91dd174468a7e71ac7a17c9a95)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 11:03:53.579237 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78351ab-f65d-4b15-8ab9-36ba1095648a admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 11:03:53.584500 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 11:03:53.626525 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-a78351ab-f65d-4b15-8ab9-36ba1095648a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cdeb6f91dd174468a7e71ac7a17c9a95'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 11:03:53.633530 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-a78351ab-f65d-4b15-8ab9-36ba1095648a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cdeb6f91dd174468a7e71ac7a17c9a95'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 11:03:53.641129 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-a78351ab-f65d-4b15-8ab9-36ba1095648a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cdeb6f91dd174468a7e71ac7a17c9a95'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 11:03:53.642069 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-a78351ab-f65d-4b15-8ab9-36ba1095648a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cdeb6f91dd174468a7e71ac7a17c9a95'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 11:03:53.642740 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-a78351ab-f65d-4b15-8ab9-36ba1095648a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cdeb6f91dd174468a7e71ac7a17c9a95'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 11:03:53.660601 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1448/2871] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Jul 27 11:03:53 2022] DELETE /identity/v3/users/cdeb6f91dd174468a7e71ac7a17c9a95 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 11:03:53.835583 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3beb58-0bbf-4254-b937-20ae45c35675 admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 11:03:53.837114 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-2f3beb58-0bbf-4254-b937-20ae45c35675 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 11:03:53.850500 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3beb58-0bbf-4254-b937-20ae45c35675 admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 11:03:53.907434 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942ecc9c-bf57-4530-b5e4-592548c607aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 11:03:53.908568 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942ecc9c-bf57-4530-b5e4-592548c607aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 11:03:53.908853 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942ecc9c-bf57-4530-b5e4-592548c607aa admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 11:03:53.909126 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942ecc9c-bf57-4530-b5e4-592548c607aa admin admin] PATH_INFO: `/v3/projects/94b763d4411f4205b7a176b6db443c9a` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 11:03:53.910271 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942ecc9c-bf57-4530-b5e4-592548c607aa admin admin] RBAC: Authorizing `identity:delete_project(project_id=94b763d4411f4205b7a176b6db443c9a)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 11:03:53.912241 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942ecc9c-bf57-4530-b5e4-592548c607aa admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 11:03:53.918276 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 11:03:53.960835 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-942ecc9c-bf57-4530-b5e4-592548c607aa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '94b763d4411f4205b7a176b6db443c9a', 'request_id': 'req-942ecc9c-bf57-4530-b5e4-592548c607aa'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 11:03:53.966910 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-942ecc9c-bf57-4530-b5e4-592548c607aa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '94b763d4411f4205b7a176b6db443c9a', 'request_id': 'req-942ecc9c-bf57-4530-b5e4-592548c607aa'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 11:03:53.974822 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-942ecc9c-bf57-4530-b5e4-592548c607aa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '94b763d4411f4205b7a176b6db443c9a', 'request_id': 'req-942ecc9c-bf57-4530-b5e4-592548c607aa'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 11:03:53.985005 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-942ecc9c-bf57-4530-b5e4-592548c607aa admin admin] The token cache is being invalidate because project 94b763d4411f4205b7a176b6db443c9a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86828) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 11:03:53.985292 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-942ecc9c-bf57-4530-b5e4-592548c607aa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 11:03:53.985905 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-942ecc9c-bf57-4530-b5e4-592548c607aa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 11:03:53.987436 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1424/2872] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Jul 27 11:03:53 2022] DELETE /identity/v3/projects/94b763d4411f4205b7a176b6db443c9a => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 11:03:53.992596 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418437bb-d77f-4ecc-adc0-92cee0c6cae2 None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 11:03:53.992956 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418437bb-d77f-4ecc-adc0-92cee0c6cae2 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 11:03:53.993329 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418437bb-d77f-4ecc-adc0-92cee0c6cae2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 11:03:54.012632 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-418437bb-d77f-4ecc-adc0-92cee0c6cae2 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 11:03:54.013825 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-418437bb-d77f-4ecc-adc0-92cee0c6cae2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 11:03:54.078867 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1449/2873] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 11:03:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 11:03:54.088078 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69a7ba6-cefc-4d8b-bd0d-3b49d5d788f1 None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 11:03:54.089222 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69a7ba6-cefc-4d8b-bd0d-3b49d5d788f1 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 11:03:54.089848 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69a7ba6-cefc-4d8b-bd0d-3b49d5d788f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 11:03:54.110649 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-f69a7ba6-cefc-4d8b-bd0d-3b49d5d788f1 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 11:03:54.112108 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-f69a7ba6-cefc-4d8b-bd0d-3b49d5d788f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 11:03:54.120197 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1425/2874] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 11:03:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 11:03:54.124479 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418437bb-d77f-4ecc-adc0-92cee0c6cae2 None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 11:03:54.136103 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418437bb-d77f-4ecc-adc0-92cee0c6cae2 None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 11:03:54.140827 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f29965-e284-4326-85a0-17a7256df999 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 11:03:54.141439 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f29965-e284-4326-85a0-17a7256df999 admin admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 11:03:54.141732 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f29965-e284-4326-85a0-17a7256df999 admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 11:03:54.142616 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f29965-e284-4326-85a0-17a7256df999 admin admin] PATH_INFO: `/v3/domains` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 11:03:54.143336 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f29965-e284-4326-85a0-17a7256df999 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86827) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 11:03:54.143635 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f29965-e284-4326-85a0-17a7256df999 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 11:03:54.145111 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f29965-e284-4326-85a0-17a7256df999 admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 11:03:54.156835 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1450/2875] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Jul 27 11:03:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 11:03:54.161088 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69a7ba6-cefc-4d8b-bd0d-3b49d5d788f1 None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 11:03:54.171020 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69a7ba6-cefc-4d8b-bd0d-3b49d5d788f1 None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 11:03:54.176701 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203bc3a9-6696-4854-89f6-abc3398f86e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 11:03:54.177345 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203bc3a9-6696-4854-89f6-abc3398f86e0 admin admin] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 11:03:54.177624 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203bc3a9-6696-4854-89f6-abc3398f86e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 11:03:54.177897 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203bc3a9-6696-4854-89f6-abc3398f86e0 admin admin] PATH_INFO: `/v3/projects` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 11:03:54.179007 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203bc3a9-6696-4854-89f6-abc3398f86e0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 11:03:54.181726 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203bc3a9-6696-4854-89f6-abc3398f86e0 admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 11:03:54.211357 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1426/2876] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Jul 27 11:03:54 2022] POST /identity/v3/projects => generated 377 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 11:03:54.215496 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f29965-e284-4326-85a0-17a7256df999 admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 11:03:54.225640 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f29965-e284-4326-85a0-17a7256df999 admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 11:03:54.281992 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f022bef4-3974-43ce-8c28-0267ac22c7df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 11:03:54.282648 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f022bef4-3974-43ce-8c28-0267ac22c7df admin admin] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 11:03:54.282934 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f022bef4-3974-43ce-8c28-0267ac22c7df admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 11:03:54.283208 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f022bef4-3974-43ce-8c28-0267ac22c7df admin admin] PATH_INFO: `/v3/users` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 11:03:54.283873 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f022bef4-3974-43ce-8c28-0267ac22c7df admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 11:03:54.285798 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f022bef4-3974-43ce-8c28-0267ac22c7df admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 11:03:54.306135 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1451/2877] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Jul 27 11:03:54 2022] POST /identity/v3/users => generated 289 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 11:03:54.310419 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203bc3a9-6696-4854-89f6-abc3398f86e0 admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 11:03:54.321702 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203bc3a9-6696-4854-89f6-abc3398f86e0 admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 11:03:54.326832 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d61561-9d76-49a4-8c34-fa68f083158b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 11:03:54.327446 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d61561-9d76-49a4-8c34-fa68f083158b admin admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 11:03:54.327730 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d61561-9d76-49a4-8c34-fa68f083158b admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 11:03:54.328008 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d61561-9d76-49a4-8c34-fa68f083158b admin admin] PATH_INFO: `/v3/roles` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 11:03:54.328704 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d61561-9d76-49a4-8c34-fa68f083158b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 11:03:54.330944 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d61561-9d76-49a4-8c34-fa68f083158b admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 11:03:54.340754 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1427/2878] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 11:03:54 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 11:03:54.344573 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f022bef4-3974-43ce-8c28-0267ac22c7df admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 11:03:54.349787 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 11:03:54.356356 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f022bef4-3974-43ce-8c28-0267ac22c7df admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 11:03:54.362176 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5986ea5e-3517-4c7e-b777-4bcd6ac2838c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 11:03:54.362834 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5986ea5e-3517-4c7e-b777-4bcd6ac2838c admin admin] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 11:03:54.363121 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5986ea5e-3517-4c7e-b777-4bcd6ac2838c admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 11:03:54.363400 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5986ea5e-3517-4c7e-b777-4bcd6ac2838c admin admin] PATH_INFO: `/v3/roles` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 11:03:54.364096 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5986ea5e-3517-4c7e-b777-4bcd6ac2838c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 11:03:54.365693 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5986ea5e-3517-4c7e-b777-4bcd6ac2838c admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 11:03:54.377035 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1452/2879] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 11:03:54 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 11:03:54.381113 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d61561-9d76-49a4-8c34-fa68f083158b admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 11:03:54.391062 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d61561-9d76-49a4-8c34-fa68f083158b admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 11:03:54.396438 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4d962f-e5a0-4981-85bc-32309c813f10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 11:03:54.397099 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4d962f-e5a0-4981-85bc-32309c813f10 admin admin] REQUEST_METHOD: `PUT` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 11:03:54.397387 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4d962f-e5a0-4981-85bc-32309c813f10 admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 11:03:54.397668 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4d962f-e5a0-4981-85bc-32309c813f10 admin admin] PATH_INFO: `/v3/projects/4da425aa3e704dfbb09d3b57755b7d76/users/7976389565d74f509807ec8aa2e9564a/roles/4c9fc0b3baed441da2126be41cc6b04a` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 11:03:54.412253 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4d962f-e5a0-4981-85bc-32309c813f10 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4da425aa3e704dfbb09d3b57755b7d76, user_id=7976389565d74f509807ec8aa2e9564a, role_id=4c9fc0b3baed441da2126be41cc6b04a)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 11:03:54.414859 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4d962f-e5a0-4981-85bc-32309c813f10 admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 11:03:54.427351 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1428/2880] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 11:03:54 2022] PUT /identity/v3/projects/4da425aa3e704dfbb09d3b57755b7d76/users/7976389565d74f509807ec8aa2e9564a/roles/4c9fc0b3baed441da2126be41cc6b04a => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 11:03:54.432309 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc36d98-ab09-49d8-9c05-a08a31e25d61 None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 11:03:54.432604 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc36d98-ab09-49d8-9c05-a08a31e25d61 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 11:03:54.432882 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc36d98-ab09-49d8-9c05-a08a31e25d61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 11:03:54.452726 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-bdc36d98-ab09-49d8-9c05-a08a31e25d61 None None] MFA Rules not processed for user `60b0ea1b90004781b58a0cbfd103ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 11:03:54.453953 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-bdc36d98-ab09-49d8-9c05-a08a31e25d61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 11:03:54.513625 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1453/2881] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 11:03:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 11:03:54.523429 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4d962f-e5a0-4981-85bc-32309c813f10 admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 11:03:54.525169 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-da4d962f-e5a0-4981-85bc-32309c813f10 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 11:03:54.581656 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4d962f-e5a0-4981-85bc-32309c813f10 admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 11:03:54.585298 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b88da68-aaa2-47db-afce-b0307eeb1926 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7922d0a1a7c4e808d0b2a80dca0da40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 11:03:54.585912 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b88da68-aaa2-47db-afce-b0307eeb1926 service neutron] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 11:03:54.586198 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b88da68-aaa2-47db-afce-b0307eeb1926 service neutron] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 11:03:54.586476 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b88da68-aaa2-47db-afce-b0307eeb1926 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 11:03:54.595741 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b88da68-aaa2-47db-afce-b0307eeb1926 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 11:03:54.597167 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b88da68-aaa2-47db-afce-b0307eeb1926 service neutron] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 11:03:54.606490 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1429/2882] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 11:03:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 11:03:58.582371 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035f5cc9-5a4b-4fe2-8da1-89aec028a3a8 None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 11:03:58.583010 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035f5cc9-5a4b-4fe2-8da1-89aec028a3a8 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 11:03:58.583010 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035f5cc9-5a4b-4fe2-8da1-89aec028a3a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 11:03:58.616913 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-035f5cc9-5a4b-4fe2-8da1-89aec028a3a8 None None] MFA Rules not processed for user `7976389565d74f509807ec8aa2e9564a`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 11:03:58.618182 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-035f5cc9-5a4b-4fe2-8da1-89aec028a3a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 11:03:58.690359 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1454/2883] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 11:03:58 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 11:03:58.694865 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b88da68-aaa2-47db-afce-b0307eeb1926 service neutron] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 11:03:58.705563 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b88da68-aaa2-47db-afce-b0307eeb1926 service neutron] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 11:03:58.711413 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e158850-5e15-492d-8449-ecf982aeafa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 11:03:58.712043 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e158850-5e15-492d-8449-ecf982aeafa2 admin admin] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 11:03:58.712324 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e158850-5e15-492d-8449-ecf982aeafa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 11:03:58.712598 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e158850-5e15-492d-8449-ecf982aeafa2 admin admin] PATH_INFO: `/v3/projects` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 11:03:58.713272 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e158850-5e15-492d-8449-ecf982aeafa2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 11:03:58.715233 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e158850-5e15-492d-8449-ecf982aeafa2 admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 11:03:58.744955 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1430/2884] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Jul 27 11:03:58 2022] POST /identity/v3/projects => generated 377 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 11:03:58.749237 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035f5cc9-5a4b-4fe2-8da1-89aec028a3a8 None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 11:03:58.761165 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035f5cc9-5a4b-4fe2-8da1-89aec028a3a8 None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 11:03:58.817890 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81078e2-22cc-49d1-933a-875469d1d0b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 11:03:58.818577 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81078e2-22cc-49d1-933a-875469d1d0b0 admin admin] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 11:03:58.818940 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81078e2-22cc-49d1-933a-875469d1d0b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 11:03:58.819220 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81078e2-22cc-49d1-933a-875469d1d0b0 admin admin] PATH_INFO: `/v3/users` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 11:03:58.819895 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81078e2-22cc-49d1-933a-875469d1d0b0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 11:03:58.821855 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81078e2-22cc-49d1-933a-875469d1d0b0 admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 11:03:58.844388 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1455/2885] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Jul 27 11:03:58 2022] POST /identity/v3/users => generated 295 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 11:03:58.848403 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e158850-5e15-492d-8449-ecf982aeafa2 admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 11:03:58.858734 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e158850-5e15-492d-8449-ecf982aeafa2 admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 11:03:58.863420 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ba79a7-42a6-431c-a59b-68923d38b383 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 11:03:58.864031 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ba79a7-42a6-431c-a59b-68923d38b383 admin admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 11:03:58.864323 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ba79a7-42a6-431c-a59b-68923d38b383 admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 11:03:58.864603 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ba79a7-42a6-431c-a59b-68923d38b383 admin admin] PATH_INFO: `/v3/roles` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 11:03:58.865302 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ba79a7-42a6-431c-a59b-68923d38b383 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 11:03:58.867192 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ba79a7-42a6-431c-a59b-68923d38b383 admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 11:03:58.872400 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 11:03:58.881747 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1431/2886] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 11:03:58 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 11:03:58.882979 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81078e2-22cc-49d1-933a-875469d1d0b0 admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 11:03:58.893484 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81078e2-22cc-49d1-933a-875469d1d0b0 admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 11:03:58.898126 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c71f8e-d95e-4820-989d-c40fe003dd89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 11:03:58.898820 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c71f8e-d95e-4820-989d-c40fe003dd89 admin admin] REQUEST_METHOD: `PUT` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 11:03:58.899107 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c71f8e-d95e-4820-989d-c40fe003dd89 admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 11:03:58.899386 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c71f8e-d95e-4820-989d-c40fe003dd89 admin admin] PATH_INFO: `/v3/projects/23891070f598436abb65c33ef92dff02/users/6b8a2855fe794b79ac88eb74a7e89dd5/roles/21156560eb6b436bb567a40b4976f3f4` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 11:03:58.907976 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 11:03:58.915443 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c71f8e-d95e-4820-989d-c40fe003dd89 admin admin] RBAC: Authorizing `identity:create_grant(project_id=23891070f598436abb65c33ef92dff02, user_id=6b8a2855fe794b79ac88eb74a7e89dd5, role_id=21156560eb6b436bb567a40b4976f3f4)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 11:03:58.918115 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c71f8e-d95e-4820-989d-c40fe003dd89 admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 11:03:58.931135 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1456/2887] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 11:03:58 2022] PUT /identity/v3/projects/23891070f598436abb65c33ef92dff02/users/6b8a2855fe794b79ac88eb74a7e89dd5/roles/21156560eb6b436bb567a40b4976f3f4 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 11:03:58.935156 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ba79a7-42a6-431c-a59b-68923d38b383 admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 11:03:58.947285 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ba79a7-42a6-431c-a59b-68923d38b383 admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 11:03:59.005541 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c395a508-f05a-49ac-a84c-4cb5d47e4481 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 11:03:59.006157 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c395a508-f05a-49ac-a84c-4cb5d47e4481 admin admin] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 11:03:59.006439 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c395a508-f05a-49ac-a84c-4cb5d47e4481 admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 11:03:59.006747 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c395a508-f05a-49ac-a84c-4cb5d47e4481 admin admin] PATH_INFO: `/v3/roles` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 11:03:59.007429 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c395a508-f05a-49ac-a84c-4cb5d47e4481 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 11:03:59.009009 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c395a508-f05a-49ac-a84c-4cb5d47e4481 admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 11:03:59.018490 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1432/2888] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 11:03:58 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 11:03:59.022680 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c71f8e-d95e-4820-989d-c40fe003dd89 admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 11:03:59.033171 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c71f8e-d95e-4820-989d-c40fe003dd89 admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 11:03:59.037774 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de66ebfe-00d7-4044-9efc-7d65e0dc5c62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 11:03:59.038433 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de66ebfe-00d7-4044-9efc-7d65e0dc5c62 admin admin] REQUEST_METHOD: `PUT` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 11:03:59.038759 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de66ebfe-00d7-4044-9efc-7d65e0dc5c62 admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 11:03:59.039041 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de66ebfe-00d7-4044-9efc-7d65e0dc5c62 admin admin] PATH_INFO: `/v3/projects/23891070f598436abb65c33ef92dff02/users/6b8a2855fe794b79ac88eb74a7e89dd5/roles/21156560eb6b436bb567a40b4976f3f4` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 11:03:59.041030 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de66ebfe-00d7-4044-9efc-7d65e0dc5c62 admin admin] RBAC: Authorizing `identity:create_grant(project_id=23891070f598436abb65c33ef92dff02, user_id=6b8a2855fe794b79ac88eb74a7e89dd5, role_id=21156560eb6b436bb567a40b4976f3f4)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 11:03:59.043709 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de66ebfe-00d7-4044-9efc-7d65e0dc5c62 admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 11:03:59.054293 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1457/2889] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 11:03:59 2022] PUT /identity/v3/projects/23891070f598436abb65c33ef92dff02/users/6b8a2855fe794b79ac88eb74a7e89dd5/roles/21156560eb6b436bb567a40b4976f3f4 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 11:04:03.568119 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc02973f-ad95-4a2c-aca2-8a343c556d20 None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 11:04:03.569102 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc02973f-ad95-4a2c-aca2-8a343c556d20 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 11:04:03.572239 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc02973f-ad95-4a2c-aca2-8a343c556d20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 11:04:03.609145 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-cc02973f-ad95-4a2c-aca2-8a343c556d20 None None] MFA Rules not processed for user `6b8a2855fe794b79ac88eb74a7e89dd5`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 11:04:03.611167 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-cc02973f-ad95-4a2c-aca2-8a343c556d20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 11:04:03.706157 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1433/2890] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 11:04:03 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 11:04:03.721475 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de66ebfe-00d7-4044-9efc-7d65e0dc5c62 admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 11:04:03.723662 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-de66ebfe-00d7-4044-9efc-7d65e0dc5c62 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 11:04:03.810492 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de66ebfe-00d7-4044-9efc-7d65e0dc5c62 admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 11:04:03.818967 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be450dc-981b-4baf-8fdb-fe0820523675 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e059d8b6c28c4fcba925d05d7c98e4ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 11:04:03.820404 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be450dc-981b-4baf-8fdb-fe0820523675 service nova] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 11:04:03.820902 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be450dc-981b-4baf-8fdb-fe0820523675 service nova] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 11:04:03.822293 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be450dc-981b-4baf-8fdb-fe0820523675 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 11:04:03.888716 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be450dc-981b-4baf-8fdb-fe0820523675 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 11:04:03.890388 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be450dc-981b-4baf-8fdb-fe0820523675 service nova] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 11:04:03.900688 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1458/2891] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 11:04:03 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 11:04:13.701377 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=63 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 11:04:13.895952 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=66,keystone:INSERT=2 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 11:04:19.136828 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24853f3-62c2-40ca-96ad-fa37dbf8b916 None None] REQUEST_METHOD: `POST` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 11:04:19.137315 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24853f3-62c2-40ca-96ad-fa37dbf8b916 None None] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 11:04:19.137458 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24853f3-62c2-40ca-96ad-fa37dbf8b916 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 11:04:19.155960 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.auth.core [None req-e24853f3-62c2-40ca-96ad-fa37dbf8b916 None None] MFA Rules not processed for user `7976389565d74f509807ec8aa2e9564a`. Rule list: `[]` (Enabled: `True`). {{(pid=86828) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 11:04:19.157156 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-e24853f3-62c2-40ca-96ad-fa37dbf8b916 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 11:04:19.163991 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1434/2892] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 11:04:19 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 11:04:29.150723 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=3 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 11:04:34.169478 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be450dc-981b-4baf-8fdb-fe0820523675 service nova] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 11:04:34.170990 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-9be450dc-981b-4baf-8fdb-fe0820523675 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 11:04:34.227751 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be450dc-981b-4baf-8fdb-fe0820523675 service nova] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 11:04:34.231538 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b4cc68-3011-4f9b-b7cb-a34ddefc0b71 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7922d0a1a7c4e808d0b2a80dca0da40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 11:04:34.232185 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b4cc68-3011-4f9b-b7cb-a34ddefc0b71 service neutron] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 11:04:34.232469 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b4cc68-3011-4f9b-b7cb-a34ddefc0b71 service neutron] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 11:04:34.232753 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b4cc68-3011-4f9b-b7cb-a34ddefc0b71 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 11:04:34.242546 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b4cc68-3011-4f9b-b7cb-a34ddefc0b71 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 11:04:34.244014 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b4cc68-3011-4f9b-b7cb-a34ddefc0b71 service neutron] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 11:04:34.253773 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1459/2893] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 11:04:34 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 11:04:44.249317 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=26 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 11:05:14.081972 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24853f3-62c2-40ca-96ad-fa37dbf8b916 None None] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 11:05:14.091924 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24853f3-62c2-40ca-96ad-fa37dbf8b916 None None] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 11:05:14.095649 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b24e94-9fab-48fb-83df-b216c1499a41 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7922d0a1a7c4e808d0b2a80dca0da40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 11:05:14.096273 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b24e94-9fab-48fb-83df-b216c1499a41 service neutron] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 11:05:14.096557 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b24e94-9fab-48fb-83df-b216c1499a41 service neutron] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 11:05:14.096836 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b24e94-9fab-48fb-83df-b216c1499a41 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 11:05:14.098648 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-a9b24e94-9fab-48fb-83df-b216c1499a41 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 11:05:14.108427 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b24e94-9fab-48fb-83df-b216c1499a41 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 11:05:14.109878 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b24e94-9fab-48fb-83df-b216c1499a41 service neutron] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 11:05:14.118121 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1435/2894] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 11:05:14 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 11:05:24.114231 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=6 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 11:06:12.806966 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b4cc68-3011-4f9b-b7cb-a34ddefc0b71 service neutron] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 11:06:12.808359 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-c5b4cc68-3011-4f9b-b7cb-a34ddefc0b71 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 11:06:12.944791 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b4cc68-3011-4f9b-b7cb-a34ddefc0b71 service neutron] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 11:06:12.949985 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee69908-f937-4e4c-a957-ada8f7311a82 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16cc6179e9a04b6398fbacb40ad1c555', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 11:06:12.950620 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee69908-f937-4e4c-a957-ada8f7311a82 service placement] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 11:06:12.950912 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee69908-f937-4e4c-a957-ada8f7311a82 service placement] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 11:06:12.951710 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee69908-f937-4e4c-a957-ada8f7311a82 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 11:06:12.953448 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-fee69908-f937-4e4c-a957-ada8f7311a82 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 11:06:12.964909 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee69908-f937-4e4c-a957-ada8f7311a82 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 11:06:12.966466 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee69908-f937-4e4c-a957-ada8f7311a82 service placement] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 11:06:12.976854 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1460/2895] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 11:06:12 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 11:06:22.972642 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=63 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 11:06:31.268910 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b24e94-9fab-48fb-83df-b216c1499a41 service neutron] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 11:06:31.270625 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-a9b24e94-9fab-48fb-83df-b216c1499a41 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 11:06:31.353698 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b24e94-9fab-48fb-83df-b216c1499a41 service neutron] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 11:06:31.364846 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea884dc5-b5d1-42a4-a756-1c03b68c1119 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e059d8b6c28c4fcba925d05d7c98e4ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 11:06:31.365497 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea884dc5-b5d1-42a4-a756-1c03b68c1119 service nova] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 11:06:31.365782 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea884dc5-b5d1-42a4-a756-1c03b68c1119 service nova] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 11:06:31.366057 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea884dc5-b5d1-42a4-a756-1c03b68c1119 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 11:06:31.367940 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-ea884dc5-b5d1-42a4-a756-1c03b68c1119 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 11:06:31.383462 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea884dc5-b5d1-42a4-a756-1c03b68c1119 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 11:06:31.385248 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea884dc5-b5d1-42a4-a756-1c03b68c1119 service nova] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 11:06:31.398020 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1436/2896] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 11:06:31 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 11:06:32.477121 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead7e865-af1d-4e2a-b99e-443fef2a08ae None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 11:06:32.477593 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead7e865-af1d-4e2a-b99e-443fef2a08ae None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 11:06:32.477724 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead7e865-af1d-4e2a-b99e-443fef2a08ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 11:06:32.496901 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-ead7e865-af1d-4e2a-b99e-443fef2a08ae None None] MFA Rules not processed for user `7976389565d74f509807ec8aa2e9564a`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 11:06:32.498819 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-ead7e865-af1d-4e2a-b99e-443fef2a08ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 11:06:32.505635 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1461/2897] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 11:06:32 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 11:06:33.503057 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea884dc5-b5d1-42a4-a756-1c03b68c1119 service nova] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 11:06:33.515351 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea884dc5-b5d1-42a4-a756-1c03b68c1119 service nova] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 11:06:33.520333 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a89862a-07f7-406f-ac17-7fc92d7e39fd service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16cc6179e9a04b6398fbacb40ad1c555', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 11:06:33.520952 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a89862a-07f7-406f-ac17-7fc92d7e39fd service placement] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 11:06:33.521390 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a89862a-07f7-406f-ac17-7fc92d7e39fd service placement] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 11:06:33.521667 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a89862a-07f7-406f-ac17-7fc92d7e39fd service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 11:06:33.523458 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-2a89862a-07f7-406f-ac17-7fc92d7e39fd service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 11:06:33.535630 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a89862a-07f7-406f-ac17-7fc92d7e39fd service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 11:06:33.537195 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a89862a-07f7-406f-ac17-7fc92d7e39fd service placement] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 11:06:33.547972 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1437/2898] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 11:06:33 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 11:06:42.491791 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=3 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 11:06:43.543524 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=25 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 11:06:46.605949 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead7e865-af1d-4e2a-b99e-443fef2a08ae None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 11:06:46.620606 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead7e865-af1d-4e2a-b99e-443fef2a08ae None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 11:06:46.626560 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd926cc-0a18-47ea-90f8-ac37370012df service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e059d8b6c28c4fcba925d05d7c98e4ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 11:06:46.627208 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd926cc-0a18-47ea-90f8-ac37370012df service nova] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 11:06:46.627491 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd926cc-0a18-47ea-90f8-ac37370012df service nova] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 11:06:46.627769 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd926cc-0a18-47ea-90f8-ac37370012df service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 11:06:46.629519 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-afd926cc-0a18-47ea-90f8-ac37370012df service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 11:06:46.642476 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd926cc-0a18-47ea-90f8-ac37370012df service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 11:06:46.644085 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd926cc-0a18-47ea-90f8-ac37370012df service nova] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 11:06:46.655503 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1462/2899] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 11:06:46 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 11:06:50.497907 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a89862a-07f7-406f-ac17-7fc92d7e39fd service placement] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 11:06:50.517771 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a89862a-07f7-406f-ac17-7fc92d7e39fd service placement] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 11:06:50.523227 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0955ff-691c-4de5-a9c6-f192a83d5191 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e059d8b6c28c4fcba925d05d7c98e4ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 11:06:50.523844 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0955ff-691c-4de5-a9c6-f192a83d5191 service nova] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 11:06:50.524127 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0955ff-691c-4de5-a9c6-f192a83d5191 service nova] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 11:06:50.524401 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0955ff-691c-4de5-a9c6-f192a83d5191 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 11:06:50.526145 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-0a0955ff-691c-4de5-a9c6-f192a83d5191 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 11:06:50.539274 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0955ff-691c-4de5-a9c6-f192a83d5191 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 11:06:50.540935 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0955ff-691c-4de5-a9c6-f192a83d5191 service nova] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 11:06:50.552158 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1438/2900] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 11:06:50 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 11:06:56.650297 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=6 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 11:07:00.547350 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=9 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 11:08:23.551596 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb30eca5-dcb5-49a9-b186-e846e31e1bb0 None None] REQUEST_METHOD: `POST` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 11:08:23.552144 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb30eca5-dcb5-49a9-b186-e846e31e1bb0 None None] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 11:08:23.552144 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb30eca5-dcb5-49a9-b186-e846e31e1bb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 11:08:23.570353 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.auth.core [None req-fb30eca5-dcb5-49a9-b186-e846e31e1bb0 None None] MFA Rules not processed for user `7976389565d74f509807ec8aa2e9564a`. Rule list: `[]` (Enabled: `True`). {{(pid=86827) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 11:08:23.571565 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-fb30eca5-dcb5-49a9-b186-e846e31e1bb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 11:08:23.578766 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1463/2901] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 11:08:23 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 11:08:23.896889 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0955ff-691c-4de5-a9c6-f192a83d5191 service nova] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 11:08:23.915919 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0955ff-691c-4de5-a9c6-f192a83d5191 service nova] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 11:08:23.919629 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d956871-30bd-4582-8690-d8e435355878 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7922d0a1a7c4e808d0b2a80dca0da40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 11:08:23.920243 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d956871-30bd-4582-8690-d8e435355878 service neutron] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 11:08:23.920526 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d956871-30bd-4582-8690-d8e435355878 service neutron] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 11:08:23.920798 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d956871-30bd-4582-8690-d8e435355878 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 11:08:23.922549 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-5d956871-30bd-4582-8690-d8e435355878 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 11:08:23.945674 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d956871-30bd-4582-8690-d8e435355878 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 11:08:23.947459 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d956871-30bd-4582-8690-d8e435355878 service neutron] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 11:08:23.956413 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1439/2902] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 11:08:23 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 11:08:33.565713 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=3 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 11:08:33.952307 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=15 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 11:10:08.986120 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb30eca5-dcb5-49a9-b186-e846e31e1bb0 None None] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 11:10:08.996138 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb30eca5-dcb5-49a9-b186-e846e31e1bb0 None None] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 11:10:08.999649 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb2dda0-9f97-47c7-8e4e-caa491c6f371 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7922d0a1a7c4e808d0b2a80dca0da40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 11:10:09.000258 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb2dda0-9f97-47c7-8e4e-caa491c6f371 service neutron] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 11:10:09.000541 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb2dda0-9f97-47c7-8e4e-caa491c6f371 service neutron] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 11:10:09.000815 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb2dda0-9f97-47c7-8e4e-caa491c6f371 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 11:10:09.009228 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb2dda0-9f97-47c7-8e4e-caa491c6f371 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 11:10:09.010699 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb2dda0-9f97-47c7-8e4e-caa491c6f371 service neutron] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 11:10:09.020373 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1464/2903] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 11:10:08 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 11:10:10.051773 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d956871-30bd-4582-8690-d8e435355878 service neutron] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 11:10:10.060574 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d956871-30bd-4582-8690-d8e435355878 service neutron] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 11:10:10.064341 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963bc584-a0a1-4462-9d58-e6c9762ef87a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7922d0a1a7c4e808d0b2a80dca0da40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 11:10:10.065004 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963bc584-a0a1-4462-9d58-e6c9762ef87a service neutron] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 11:10:10.065288 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963bc584-a0a1-4462-9d58-e6c9762ef87a service neutron] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 11:10:10.065566 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963bc584-a0a1-4462-9d58-e6c9762ef87a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 11:10:10.067335 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-963bc584-a0a1-4462-9d58-e6c9762ef87a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 11:10:10.095047 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963bc584-a0a1-4462-9d58-e6c9762ef87a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 11:10:10.097055 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963bc584-a0a1-4462-9d58-e6c9762ef87a service neutron] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 11:10:10.118545 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1440/2904] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 11:10:10 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 11:10:11.311174 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb2dda0-9f97-47c7-8e4e-caa491c6f371 service neutron] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 11:10:11.319835 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb2dda0-9f97-47c7-8e4e-caa491c6f371 service neutron] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 11:10:11.323560 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b303cbab-c18f-4f5d-b490-ce48324a80d9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7922d0a1a7c4e808d0b2a80dca0da40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 11:10:11.324167 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b303cbab-c18f-4f5d-b490-ce48324a80d9 service neutron] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 11:10:11.324451 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b303cbab-c18f-4f5d-b490-ce48324a80d9 service neutron] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 11:10:11.324728 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b303cbab-c18f-4f5d-b490-ce48324a80d9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 11:10:11.326506 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-b303cbab-c18f-4f5d-b490-ce48324a80d9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 11:10:11.340843 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b303cbab-c18f-4f5d-b490-ce48324a80d9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 11:10:11.342255 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b303cbab-c18f-4f5d-b490-ce48324a80d9 service neutron] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 11:10:11.353115 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1465/2905] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 11:10:11 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 11:10:16.728491 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963bc584-a0a1-4462-9d58-e6c9762ef87a service neutron] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 11:10:16.738245 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963bc584-a0a1-4462-9d58-e6c9762ef87a service neutron] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 11:10:16.742741 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0def29-9ed5-4b90-b900-1e4f0c971745 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7922d0a1a7c4e808d0b2a80dca0da40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 11:10:16.743628 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0def29-9ed5-4b90-b900-1e4f0c971745 service neutron] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 11:10:16.743932 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0def29-9ed5-4b90-b900-1e4f0c971745 service neutron] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 11:10:16.744391 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0def29-9ed5-4b90-b900-1e4f0c971745 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 11:10:16.754550 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0def29-9ed5-4b90-b900-1e4f0c971745 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 11:10:16.756519 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0def29-9ed5-4b90-b900-1e4f0c971745 service neutron] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 11:10:16.766350 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1441/2906] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 11:10:16 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 11:10:21.348339 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=12 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 11:10:22.122024 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b303cbab-c18f-4f5d-b490-ce48324a80d9 service neutron] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 11:10:22.131000 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b303cbab-c18f-4f5d-b490-ce48324a80d9 service neutron] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 11:10:22.136139 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97e4b3f-0beb-4690-bf8a-f2a494281a4a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7922d0a1a7c4e808d0b2a80dca0da40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 11:10:22.137074 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97e4b3f-0beb-4690-bf8a-f2a494281a4a service neutron] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 11:10:22.137505 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97e4b3f-0beb-4690-bf8a-f2a494281a4a service neutron] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 11:10:22.137922 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97e4b3f-0beb-4690-bf8a-f2a494281a4a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 11:10:22.213995 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97e4b3f-0beb-4690-bf8a-f2a494281a4a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 11:10:22.215467 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97e4b3f-0beb-4690-bf8a-f2a494281a4a service neutron] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 11:10:22.234326 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1466/2907] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 11:10:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 11:10:25.273825 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0def29-9ed5-4b90-b900-1e4f0c971745 service neutron] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 11:10:25.291278 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0def29-9ed5-4b90-b900-1e4f0c971745 service neutron] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 11:10:25.303922 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b134f0-1300-4971-9eee-6adf74b3514e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16cc6179e9a04b6398fbacb40ad1c555', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 11:10:25.304572 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b134f0-1300-4971-9eee-6adf74b3514e service placement] REQUEST_METHOD: `GET` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 11:10:25.305053 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b134f0-1300-4971-9eee-6adf74b3514e service placement] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 11:10:25.305338 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b134f0-1300-4971-9eee-6adf74b3514e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 11:10:25.307191 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-94b134f0-1300-4971-9eee-6adf74b3514e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 11:10:25.321018 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b134f0-1300-4971-9eee-6adf74b3514e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 11:10:25.323166 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b134f0-1300-4971-9eee-6adf74b3514e service placement] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 11:10:25.345770 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1442/2908] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 11:10:25 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 74 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 11:10:28.597583 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97e4b3f-0beb-4690-bf8a-f2a494281a4a service neutron] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 11:10:28.615131 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97e4b3f-0beb-4690-bf8a-f2a494281a4a service neutron] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 11:10:28.620597 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f765a3-c5cf-49b2-9748-64dfe8409da1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 11:10:28.621374 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f765a3-c5cf-49b2-9748-64dfe8409da1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 11:10:28.621808 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f765a3-c5cf-49b2-9748-64dfe8409da1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 11:10:28.622213 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f765a3-c5cf-49b2-9748-64dfe8409da1 admin admin] PATH_INFO: `/v3/users/7976389565d74f509807ec8aa2e9564a` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 11:10:28.623778 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f765a3-c5cf-49b2-9748-64dfe8409da1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7976389565d74f509807ec8aa2e9564a)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 11:10:28.625884 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f765a3-c5cf-49b2-9748-64dfe8409da1 admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 11:10:28.679312 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-77f765a3-c5cf-49b2-9748-64dfe8409da1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7976389565d74f509807ec8aa2e9564a'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 11:10:28.686119 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-77f765a3-c5cf-49b2-9748-64dfe8409da1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7976389565d74f509807ec8aa2e9564a'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 11:10:28.694746 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-77f765a3-c5cf-49b2-9748-64dfe8409da1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7976389565d74f509807ec8aa2e9564a'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 11:10:28.695377 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-77f765a3-c5cf-49b2-9748-64dfe8409da1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7976389565d74f509807ec8aa2e9564a'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 11:10:28.696006 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-77f765a3-c5cf-49b2-9748-64dfe8409da1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7976389565d74f509807ec8aa2e9564a'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 11:10:28.713659 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1467/2909] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Jul 27 11:10:28 2022] DELETE /identity/v3/users/7976389565d74f509807ec8aa2e9564a => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 11:10:28.893486 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b134f0-1300-4971-9eee-6adf74b3514e service placement] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 11:10:28.906034 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b134f0-1300-4971-9eee-6adf74b3514e service placement] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 11:10:28.964928 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63402f34-9840-4f4a-bf75-310f1c3a4d67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 11:10:28.965571 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63402f34-9840-4f4a-bf75-310f1c3a4d67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 11:10:28.965855 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63402f34-9840-4f4a-bf75-310f1c3a4d67 admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 11:10:28.966132 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63402f34-9840-4f4a-bf75-310f1c3a4d67 admin admin] PATH_INFO: `/v3/projects/4da425aa3e704dfbb09d3b57755b7d76` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 11:10:28.967312 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63402f34-9840-4f4a-bf75-310f1c3a4d67 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4da425aa3e704dfbb09d3b57755b7d76)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 11:10:28.969257 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63402f34-9840-4f4a-bf75-310f1c3a4d67 admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 11:10:29.015483 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-63402f34-9840-4f4a-bf75-310f1c3a4d67 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4da425aa3e704dfbb09d3b57755b7d76', 'request_id': 'req-63402f34-9840-4f4a-bf75-310f1c3a4d67'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 11:10:29.021521 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-63402f34-9840-4f4a-bf75-310f1c3a4d67 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4da425aa3e704dfbb09d3b57755b7d76', 'request_id': 'req-63402f34-9840-4f4a-bf75-310f1c3a4d67'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 11:10:29.028511 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-63402f34-9840-4f4a-bf75-310f1c3a4d67 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4da425aa3e704dfbb09d3b57755b7d76', 'request_id': 'req-63402f34-9840-4f4a-bf75-310f1c3a4d67'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 11:10:29.051840 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-63402f34-9840-4f4a-bf75-310f1c3a4d67 admin admin] The token cache is being invalidate because project 4da425aa3e704dfbb09d3b57755b7d76 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86828) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 11:10:29.052132 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-63402f34-9840-4f4a-bf75-310f1c3a4d67 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 11:10:29.052747 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-63402f34-9840-4f4a-bf75-310f1c3a4d67 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 11:10:29.054227 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1443/2910] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Jul 27 11:10:28 2022] DELETE /identity/v3/projects/4da425aa3e704dfbb09d3b57755b7d76 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 11:10:29.058217 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f765a3-c5cf-49b2-9748-64dfe8409da1 admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 11:10:29.059739 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-77f765a3-c5cf-49b2-9748-64dfe8409da1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 11:10:29.072708 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f765a3-c5cf-49b2-9748-64dfe8409da1 admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 11:10:29.117159 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 11:10:29.131585 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a53c2af-1cbf-44bb-88ec-5eb22e57690d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 11:10:29.132223 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a53c2af-1cbf-44bb-88ec-5eb22e57690d admin admin] REQUEST_METHOD: `DELETE` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 11:10:29.132505 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a53c2af-1cbf-44bb-88ec-5eb22e57690d admin admin] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 11:10:29.132774 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a53c2af-1cbf-44bb-88ec-5eb22e57690d admin admin] PATH_INFO: `/v3/users/6b8a2855fe794b79ac88eb74a7e89dd5` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 11:10:29.133929 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a53c2af-1cbf-44bb-88ec-5eb22e57690d admin admin] RBAC: Authorizing `identity:delete_user(user_id=6b8a2855fe794b79ac88eb74a7e89dd5)` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 11:10:29.135925 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a53c2af-1cbf-44bb-88ec-5eb22e57690d admin admin] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 11:10:29.182711 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-8a53c2af-1cbf-44bb-88ec-5eb22e57690d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6b8a2855fe794b79ac88eb74a7e89dd5'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 11:10:29.189722 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-8a53c2af-1cbf-44bb-88ec-5eb22e57690d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6b8a2855fe794b79ac88eb74a7e89dd5'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 11:10:29.196785 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-8a53c2af-1cbf-44bb-88ec-5eb22e57690d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6b8a2855fe794b79ac88eb74a7e89dd5'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 11:10:29.197445 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-8a53c2af-1cbf-44bb-88ec-5eb22e57690d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6b8a2855fe794b79ac88eb74a7e89dd5'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 11:10:29.198046 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.notifications [None req-8a53c2af-1cbf-44bb-88ec-5eb22e57690d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6b8a2855fe794b79ac88eb74a7e89dd5'} {{(pid=86827) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 11:10:29.216720 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1468/2911] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Jul 27 11:10:29 2022] DELETE /identity/v3/users/6b8a2855fe794b79ac88eb74a7e89dd5 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 11:10:29.386334 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63402f34-9840-4f4a-bf75-310f1c3a4d67 admin admin] Authenticating user token {{(pid=86828) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 11:10:29.387814 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.fernet_utils [None req-63402f34-9840-4f4a-bf75-310f1c3a4d67 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86828) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 11:10:29.403949 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63402f34-9840-4f4a-bf75-310f1c3a4d67 admin admin] Validating token access rules against request {{(pid=86828) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 11:10:29.423652 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 11:10:29.463516 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4f847c-f225-4fb2-ab79-fa66dcfb1b9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60b0ea1b90004781b58a0cbfd103ae8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeee704de4024dd1b602d1a289cba407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86828) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 11:10:29.464151 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4f847c-f225-4fb2-ab79-fa66dcfb1b9f admin admin] REQUEST_METHOD: `DELETE` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 11:10:29.464432 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4f847c-f225-4fb2-ab79-fa66dcfb1b9f admin admin] SCRIPT_NAME: `/identity` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 11:10:29.464708 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4f847c-f225-4fb2-ab79-fa66dcfb1b9f admin admin] PATH_INFO: `/v3/projects/23891070f598436abb65c33ef92dff02` {{(pid=86828) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 11:10:29.465863 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4f847c-f225-4fb2-ab79-fa66dcfb1b9f admin admin] RBAC: Authorizing `identity:delete_project(project_id=23891070f598436abb65c33ef92dff02)` {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 11:10:29.467790 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4f847c-f225-4fb2-ab79-fa66dcfb1b9f admin admin] RBAC: Authorization granted {{(pid=86828) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 11:10:29.512620 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-6c4f847c-f225-4fb2-ab79-fa66dcfb1b9f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '23891070f598436abb65c33ef92dff02', 'request_id': 'req-6c4f847c-f225-4fb2-ab79-fa66dcfb1b9f'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 11:10:29.519091 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-6c4f847c-f225-4fb2-ab79-fa66dcfb1b9f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '23891070f598436abb65c33ef92dff02', 'request_id': 'req-6c4f847c-f225-4fb2-ab79-fa66dcfb1b9f'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 11:10:29.527025 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-6c4f847c-f225-4fb2-ab79-fa66dcfb1b9f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '23891070f598436abb65c33ef92dff02', 'request_id': 'req-6c4f847c-f225-4fb2-ab79-fa66dcfb1b9f'} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 11:10:29.538010 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-6c4f847c-f225-4fb2-ab79-fa66dcfb1b9f admin admin] The token cache is being invalidate because project 23891070f598436abb65c33ef92dff02 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86828) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 11:10:29.538234 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-6c4f847c-f225-4fb2-ab79-fa66dcfb1b9f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 11:10:29.538873 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG keystone.notifications [None req-6c4f847c-f225-4fb2-ab79-fa66dcfb1b9f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86828) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 11:10:29.540375 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: [pid: 86828|app: 0|req: 1444/2912] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Jul 27 11:10:29 2022] DELETE /identity/v3/projects/23891070f598436abb65c33ef92dff02 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 11:10:39.213334 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=31,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 11:10:39.535003 dv-cinder-850840-3-iscsi devstack@keystone.service[86828]: DEBUG dbcounter [-] [86828] Writing DB stats keystone:SELECT=42,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86828) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 11:11:21.942364 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a53c2af-1cbf-44bb-88ec-5eb22e57690d admin admin] Authenticating user token {{(pid=86827) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 11:11:21.943940 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-8a53c2af-1cbf-44bb-88ec-5eb22e57690d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 11:11:22.012270 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a53c2af-1cbf-44bb-88ec-5eb22e57690d admin admin] Validating token access rules against request {{(pid=86827) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 11:11:22.068972 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151ec4fb-e17d-4226-85b2-8740e99a7686 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16cc6179e9a04b6398fbacb40ad1c555', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f296d8e3b8f4001909addc49d8dac01', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86827) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 11:11:22.069586 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151ec4fb-e17d-4226-85b2-8740e99a7686 service placement] REQUEST_METHOD: `GET` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 11:11:22.070587 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151ec4fb-e17d-4226-85b2-8740e99a7686 service placement] SCRIPT_NAME: `/identity` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 11:11:22.070748 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151ec4fb-e17d-4226-85b2-8740e99a7686 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86827) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 11:11:22.072434 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.fernet_utils [None req-151ec4fb-e17d-4226-85b2-8740e99a7686 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86827) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 11:11:22.083654 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151ec4fb-e17d-4226-85b2-8740e99a7686 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 11:11:22.085216 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151ec4fb-e17d-4226-85b2-8740e99a7686 service placement] RBAC: Authorization granted {{(pid=86827) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 11:11:22.095602 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: [pid: 86827|app: 0|req: 1469/2913] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 11:11:21 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 11:11:32.091432 dv-cinder-850840-3-iscsi devstack@keystone.service[86827]: DEBUG dbcounter [-] [86827] Writing DB stats keystone:SELECT=58 {{(pid=86827) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}